The command submitty_count_ts
can be used to count and verify that student submissions for a
programming course contain specific static language features.
submitty_count_ts
is a refactor and replacement of the existing haskell implementation of the submitty_count
utility. Instructors should switch to the submitty_count_ts
command and revised command line arguments in preparation for the deprecation of submitty_count
.
To use submitty_count_ts
, simply invoke it as a command within the config.json
file for a
given assignment, supplying the type of feature to count, the feature itself and
any number of source files.
Usage of the command:
submitty_count_ts -l <language> <countable> <feature> <files>...
Python
aspython
or py
, C
as c
and C++
as cpp
are accepted as languages.
Countable
could be node
, call
, function
or identifier
. The uses of each countable is described
in the below sections.
Example:
submitt_count_ts -l py call print *.py
This example will output the number of calls to the function print
in all
of the Python source files in the current directory.
Note: submitty_count_ts
is an alias for a program installed on the
submitty server. You can directly run the command to see how it works.
Here’s the same example:
/usr/local/submitty/SubmittyAnalysisToolsTS/build/submitty_count_ts -l python call print *.py
Another example:
submitty_count_ts -l cpp node goto main.cpp
This second example will output the number of occurrences of the node goto
in the
C/C++ source file main.cpp
.
Countable Features
Currently, four feature types can be counted: nodes, function definitions, identifiers, and function calls.
Nodes
The submitty_count_ts
command translates the textual student submission into a tree structure. Each node in this tree has a type and this type can be passed as the feature that needs to be counted to the submitty_count_ts
command.
The nodes contained in the tree generated from a student submission can be viewed using submitty_diagnostics_ts
command.
Usage of the command is as follows
submitty_diagnostics_ts -l <language> <files>...
This tool outputs JSON data containing the type of each node in the tree and respective portion of the student submission corresponding to each the node.
For example, if you would like to count additions, but are unsure of which node type to count, you could use a test file like:
# file.py
print(1 + 1)
Running submitty_diagnostics_ts -l python file.py
on this file will produce the following output:
{
"test.py": [
{
"end_col": 13,
"end_line": 1,
"node": "call",
"start_col": 1,
"start_line": 1
},
{
"end_col": 6,
"end_line": 1,
"node": "identifier",
"start_col": 1,
"start_line": 1
},
{
"end_col": 13,
"end_line": 1,
"node": "argument_list",
"start_col": 6,
"start_line": 1
},
{
"end_col": 7,
"end_line": 1,
"node": "(",
"start_col": 6,
"start_line": 1
},
{
"end_col": 12,
"end_line": 1,
"node": "binary_operator",
"start_col": 7,
"start_line": 1
},
{
"end_col": 8,
"end_line": 1,
"node": "integer",
"start_col": 7,
"start_line": 1
},
{
"end_col": 10,
"end_line": 1,
"node": "+",
"start_col": 9,
"start_line": 1
},
{
"end_col": 12,
"end_line": 1,
"node": "integer",
"start_col": 11,
"start_line": 1
},
{
"end_col": 13,
"end_line": 1,
"node": ")",
"start_col": 12,
"start_line": 1
}
]
}
The node
attribute in an JSON object corresponds to the node type and could be used to identify the type of the node that needs to be specified as the feature to the submitty_count_ts
command.
Notice that a node +
is present between two integer
nodes.
You could verify that this is the right node by looking at the start_line
, end_line
, start_col
, and end_col
fields, which indicate on what row and column the nodes begin and end within the file.
Once you are sure that the node is correct, you could count it within student submissions with submitty_count_ts
:
submitty_count_ts -l python node + *.py
Note: submitty_diagnostics_ts
is an alias for a program installed on the
submitty server. You can directly run the command to see how it works.
Here’s the same example:
/usr/local/submitty/SubmittyAnalysisToolsTS/build/submitty_diagnostics_ts -l python *.py
Function Calls
This method could be used to count calls to a function with a particular name.
For example to count function calls to print
function in a Python program:
submitty_count_ts -l py call print *.py
This method could be used to count function calls to functions within a particular library, class or namespace.
For example to count calls to vector.erase()
function in C++ standard template library:
submitty_count_ts -l cpp call erase *.cpp
But this may give false positives if the user has a defined function called erase
and this
function is called within the submitted code.
Function Definitions
This method can be used to count definitions of a function with a particular name.
For example to count definitions of a function named sum
in a C program:
submitty_count_ts -l c function sum *.c
Identifiers
This method can be used to count the uses of an identifier with a particular name in student code.
For example to count uses of identifier named height
in a C++ program:
submitty_count_ts -l cpp identifier height *.cpp