Information for Contestants

Kattis is an extremely popular site for programmers to hone and test their algorithmic coding skills, solving problems of all levels of complexity, and designed for individuals just having some spare time, as well as for hosting timed programming contests for up to hundreds of teams. If Kattis is new to you, check it out at https://www.kattis.com/developers. The NCNA Regional collegiate programming contest uses Kattis for its annual 5-hour 3-member team competitions.

For an NCNA Regional Contest, your physical contest site will provide you with login credentials and the URL for the competition on Kattis.  At the start of the contest you will be presented with around a dozen problems which your team may attempt to solve in any order. The team solving the most problems wins, with various strategies for resolving ties. You may write your code in your choice of Java, C++, C, Kotlin, or Python 3 which you then submit to Kattis to judge. Kattis will compile your code and run it on some secret input data. Very shortly, you will get a judgement informing you whether your code behaved as expected or not. You do get multiple chances to get it right.

You will find the problems both in the Kattis web page under the heading PROBLEMS and on paper copies provided to you at your site. The end of the online version of a problem includes a link to files containing the sample input data.  These files are a good start for generating your own more robust sets of testing data.

During the contest you can submit requests for clarifications to the judges. To do so, click on Clarifications. The clarifications page has three sections:

• submitted clarification requests from your team that have not yet been answered
• a form for submitting a clarification request
• clarification requests with answers from the judges.

When you submit a clarification request, select a subject (either one of the problems or “general”) and write your request in English. The third section contains answers to your requests and sometimes answers to other teams’ requests, in case the judges choose to reveal the question and the answer to all teams. Notifications that there are new clarification replies are displayed on all Kattis web pages, but not on other pages (e.g., API documentation and scoreboards).

Choose the Submit button at the top of the Kattis web interface.   A form appears with places to select the Problem ID and Language and a place to drag or choose files to upload.  Typically, you upload just a single source file that you have thoroughly tested, though multiple files are allowed.  The root of the file name does not have to match the problem ID. Java users: You must also enter the Main Class – the name of the class containing the main method.  Caution:  This is not the file name but the class name.  If your file hello.java has the class hello containing your main method, you indicate that the Main Class is just hello.  If you misspell this, you will get a compiler error saying you have an “Unknown main class”.

First, Kattis will compile your program. If the compiler fails to compile your program, or does not complete in a reasonable amount of time, Kattis will judge it as Compile Error. Otherwise, Kattis will execute the compiled binary on the first set of secret judge’s input (there may be several). If the execution takes too long it will be judged as Time Limit Exceeded. If it produces way too much output, it will be judged as Output Limit Exceeded. If it crashes, terminates with non-zero exit status, or uses too much memory, it will be judged as a RunTime Error. If the execution terminates correctly (exit status 0), Kattis will inspect the output produced to verify that it is correct. If it is incorrect, Kattis will judge the submission as Wrong Answer. These judgments will be discussed in more detail below.

Note that your submission’s output is only inspected if it successfully terminates in time, thus a Time Limit Exceeded or Run Time Error does not mean that output was correct up until the point where the program crashed or time ran out. Note that if your program should, for instance, divide by zero after the time limit has been reached, it will be judged as Time Limit Exceeded rather than Run Time Error. If there are multiple test files, Kattis will continue with the same procedure for the next set of judge’s test data, as long as no error has been found. As soon as an error is detected Kattis will stop and report that error. Each test is run with a new invocation of your program, so your program does not need to be adapted in any way to handle multiple test files. Each test file will follow the input specification for the problem. If your program passes all test files successfully, it will be judged as Accepted.

You can track the status of your submission in the web interface by choosing your team name from the top right menu. On this page you will see a list of all submissions you have made, in reverse chronological order. As the submission proceeds through the judgement process your submissions page will reflect this. The states a submission will pass through while it is being judged are:

1. New
2. Waiting for Compile
3. Compiling
4. Waiting for Run
5. Running
6. Final Judgement

When Kattis has judged your submission, you will get a reply telling you the status of your submission. The following judgements are possible (these are described in more detail below):

• Accepted
• Compile Error
• Run Time Error
• Time Limit Exceeded
• Output Limit Exceeded
• Wrong Answer

For compile errors, there will be extra information apart from the judgement itself, to help you debug your code syntax. This information will be available on the page for the respective submission (available by clicking on the corresponding submission ID number in your list of submissions).If your program compiles successfully, Kattis starts on a sequence of runs with secret judges’ data. Kattis stops and reports the first error found.  If a run completes normally, the output for that run is tested.  Kattis only goes on to further test runs if the last run’s output was correct.

Accepted: Accepted means that we were very happy with your program, and that it (as far as Kattis could tell) solved the problem correctly. Congratulations!

Compile Error: Compile Error means that Kattis failed to compile your source code. In order to help you debug the error, the compiler output will be available as extra information. This error does not result in a 20 minute time penalty. (You do waste the time spent submitting it!)  Information about what compilers and flags are used can be found below. Common sources of errors in Java submissions include:

• Misnaming the Main Class, leading to an error “Unknown main class”; you must specify the name of the class that contains the starting main method
• The class name contains no .java
• Putting the class in a package and entering the plain class name; then you also get the error “Unknown main class”

Run Time Error: Run Time Error means your program crashed during execution with the judges’ secret test input. More precisely it means that it terminated with a non-zero exit code, or with an uncaught exception.  Common examples are:

• Array out-of-bounds references
• Stack overflows (likely infinite recursion)
• Using too much memory
• Invalid memory references (e.g., dereferencing a null pointer)
• Trying to open a file

This last error happens if you mistakenly try to open an input or output file for reading or writing data. Remember to just use standard input and output as discussed earlier. Note that since the exit code is used to determine normal termination, it is important that your main function in a C or C++ program does not return a non-zero value.

Time Limit Exceeded: Time Limit Exceeded means that your program ran for too long. When the time limit is exceeded, the program is terminated. The output produced is not inspected until your program has finished successfully, so getting Time Limit Exceeded does not mean that the output you had produced thus far was correct.  The time limit is applied separately to each run with judges’ secret test data.  Usually there are multiple test runs before a submission is accepted. Each individual problem lists a time limit that is the limit on the web server, not your local machine. You should be given a list of approximate time limits appropriate for your tests on the hardware used at your site.

Output Limit Exceeded: Output Limit Exceeded means that your program has produced too much output and Kattis decided to shoot it down before it flooded the hard drive. Check to make sure that you don’t get stuck in an infinite loop in which you print something repeatedly. This could happen if you handle input termination incorrectly.

Wrong Answer: Wrong Answer means that your program finished within the time limit, but that the answer produced was incorrect. This error is usually the most frustrating one, since typically no extra information will be given. Sometimes, the only way around it is to try to find bugs in your code by constructing tricky test data yourself. Note that it is possible to have a technically correct solution, but get a Wrong Answer because you formatted the output incorrectly. While problems are designed to minimize the possibility of such formatting errors. Note the following:

• Only if text formatting is explicitly given as a part of the problem, will formatting variations likely cause a Wrong Answer.
• Where formatting is not a part of the problem (the most common case), then the system allows variations in white space. Problems are not likely to require much typo-prone canned text in an answer.  The output validator typically looks at the output as a sequence of white-space separated tokens.
• Instructions describing floating point answers are likely to have an error bound included, so a rounded answer with an exact textual match of digits is not required in your output.

All the errors that come after successful compilation do cost a 20-minute time penalty, assuming a later submission of the problem is accepted. Of course, getting a problem right is always preferable regardless the number of penalty minutes.

See the Kattis documentation for the exact compiler flags that will be used to compile your submission, as well as how it will be run:

• C
• C++
• Java
• Python 3
• Kotlin

System Libraries: You are allowed to use all standard libraries included with the language. Note that even if your site’s computers have a given library installed on them, that does not mean it will be available on Kattis. Most notably, Kattis does not support Python libraries like Scipy, Numpy, and Pandas.