VM Install using Vagrant

The instructions below will setup an instance of Submitty on your own hardware that will have several courses, many sample assignments, and a hundred students with assignment submissions so you can explore the features of Submitty as it would appear “mid-semester”. Your host computer can run any modern operating system (Windows, Mac, or Unix/Linux). The installation process will create a new Virtual Machine (VM) on your computer and the VM will use the Ubuntu GNU/Linux operating system.

Note: We only officially support and test development using VirtualBox. The instructions below are for VirtualBox. While alternatively using VMWare should work, we have not tested this, and do not provide these instructions.

IMPORTANT NOTE: If you are using an Apple Mac computer with Apple Silicon (e.g., M1 or M2), first released in late 2020, you will follow the Vagrant QEMU instructions. If you using an Intel-based Mac, you will follow the instructions below.

Pre-Installation Checklist

To develop with a Virtual Machine (VM), your computer should have at least 8GB of RAM and a 64-bit host OS. AMD-V or Intel VT-x are also required (most computers have these). Submitty is RAM and I/O intensive, so more RAM and a fast disk are better.
Make sure you have at least 65GB of hard disk available for installation. We do not recommend installing the Submitty Developer VM on DropBox, OneDrive, GoogleDrive, or other cloud storage.
Some developers have had problems running both VirtualBox and VMWare on the same computer. If you have problems, we suggest shutting down the VMWare VMs, or stopping the VMWare services, or uninstalling VMWare.
If you’re running Windows, it is recommended to disable Hyper-V. Leaving it enabled will force VirtualBox to use the Hyper-V backend, which will be slower and can cause instability in the VM.

Note: This may stop programs like Docker Desktop and WSL 2 from working. If these programs are essential to your workflow, consider looking up how to add a separate boot entry with “hypervisorlaunchtype” set to “off” for use with VirtualBox.

Note: Installing WSL2 may also reconfigure your OS to use Hyper-V or Windows hypervisor platform and prevent VirtualBox from working correctly. It is recommended to not install or use WSL2 alongside Virtualbox for now.
The complete installation process could take an hour or more. Make sure your internet connection is strong and consistent. You’ll probably want to plug in your laptop power cord. Check your computer settings and make sure the machine does not hibernate or go to sleep during installation.

Submitty Developer VM Installation

Enable Virtualization

MacOS
1. Virtualization is generally enabled by default.
Windows 10
1. Open the Settings app by searching for it in the windows bar or clicking it in the Windows menu.
2. Navigate to Update and Security, then select Recovery from the side menu.
3. Under Advanced Startup, click Restart Now.
4. Once your PC has rebooted, click the Troubleshoot option.
5. Click Advanced Options.
6. Click UEFI Firmware Settings and restart as suggested.
7. Enter your BIOS (generally by pressing Del, F12, or other keys while booting). If you are not able to find the key combo needed to enter your BIOS, refer to this guide.
8. Locate Virtualization, and enable it. (Note: If you cannot find the option to enable virtualization, search Google for a tutorial on enabling it with your motherboard.)
9. Reboot your computer.
Ubuntu
1. Enter your BIOS (generally by pressing Del, F12, or other keys while booting).
2. Navigate the BIOS Settings.
3. Locate Virtualization and enable it.
4. Be sure to choose Hardware Virtualization in the System -> Acceleration settings of the virtual machine you are using.
NOTE If using secure boot, vagrant may fail to work with VirtualBox. You will then either need to disable secure boot from the boot menu/BIOS or follow these steps to self-sign the necessary packages to run vagrant and VirtualBox.
Download and install the latest version of Ruby.
Download and install the latest version of Git.
Download and install VirtualBox and Vagrant

NOTE Please download VirtualBox 6 instead of 7.

Below are quick steps to get everything installed and running.

Windows 10

You can just go to the respective sites and download the necessary binaries.

MacOS

You can either go to respective sites and download the necessary binaries or install homebrew if you don’t have it and then run:
```
brew install --cask virtualbox
brew install --cask vagrant
```

Ubuntu/Debian

The Ubuntu repository does not contain the latest version of Vagrant or VirtualBox and using them may not work nor are they supported. We recommend that you either download the necessary binaries from their respective steps or follow the steps outlined below for each:

VirtualBox: https://www.virtualbox.org/wiki/Linux_Downloads

Vagrant: https://developer.hashicorp.com/vagrant/downloads (if that doesn’t work, try: https://vagrant-deb.linestarve.com/)

Clone the Submitty repository to a location on your computer (the “host”).
```
git clone https://github.com/Submitty/Submitty.git
```
NOTE: If you are not currently part of the Submitty organization on Github, you may want to fork the repo and use the git url from your fork instead, especially if you are looking to contribute.

OPTIONAL: If you will be developing code in one of the companion Submitty repositories (e.g., AnalysisTools, Lichen, Localization, RainbowGrades, Tutorial), also clone those repositories to the same directory. For example:
```
  home
  └── myusername
      └── Submitty
          └── GIT_CHECKOUT
              ├── AnalysisTools  (optional)
              ├── Lichen         (optional)
              ├── Localization   (optional)
              ├── RainbowGrades  (optional)
              ├── Submitty
              └── Tutorial       (optional)
```
This host directory structure will be shared / synced between your host operating system and the Submitty virtual machine.
Navigate into the Submitty repository on your computer in a shell/terminal and type:

Windows should run CMD or powershell on administrator mode
```
vagrant up
```
Vagrant will build your VM. This will take maybe 30 minutes to a few hours depending on your Internet connection speed. When this command finishes, your VM is ready to use.
When the vagrant up command completes successfully, you will be able to access the Submitty website (instructions follow in the next section).

The VM will continue to run jobs in the background and consume a nontrivial amount of CPU resources, while completing a backlog of autograding for a dozen or more sample submissions for each of the more than 100 users in the sample courses.

On MacOS and linux, if your development work will not require sample assignment submissions or autograding results, you may prepend NO_SUBMISSIONS=1 to the previous command, which will skip the creation of these sample submissions and their autograding and decrease the time to complete installation.
```
NO_SUBMISSIONS=1 vagrant up
```
On Windows, you will have to first set the environment variable NO_SUBMISSIONS to 1 which lasts for the session of that console, then call vagrant up.
```
SET NO_SUBMISSIONS=1
vagrant up
```
If you want to unset the variable later, you can do
```
SET NO_SUBMISSIONS=
```
Similarly, you can check that the variable is set by doing
```
SET NO_SUBMISSIONS
```

When the install has completed, you should see the message:

#####################################################################

                     INSTALLATION SUCCESS!
   
                        .GGQGGGSlu
                      .GGGGGGGGGGGS
                 :llUGGGGGGGGGGGGGGGG
                 'GGGGGGGGGGGGGGGGGGb        .
                    %GGGGGGGGGGGGGGG~   ..GSGGG
                       GGGGGGGGGGGGGGSGGGGGGGGGG[
                     ;GGGGGGGGGGGGp\ \ \GGGGGGGGL
                    !GGGGGGGGGGGGGGS\ \ \GGGGGG
                    GGGGGGGGGGGGGGGGG\ \ \9GGGG
                    %GGGGGGGGGGGGGGGS/ / /.GGG
                     %GGGGGGGGGGGGGS/ / /GGG
                      '%NNNNNNNNNNNNNNNNNN
#####################################################################

NOTE: There are times when the install will pause for a brief period with the message Done. This does not mean the install has ended, and the install should continue after a bit of time.

If you do not see this message due to an error or the installation has frozen, check out:

Using your Submitty Developer VM

When the VM is “up”, you can go visit the homework submission website.
- From a web browser (Chrome, Firefox, IE, etc.) on your host computer, go to:
  
  http://localhost:1511/index.php
  
  (see the VM login & password info below)
- You can test the submission, autograding, and viewing of the grades details by uploading sample submissions from the Submitty repository, located in one of these these directories:
  
  For the “tutorial” course:
  https://github.com/Submitty/Tutorial/tree/main/examples
  
  For the “sample” course:
  https://github.com/Submitty/Submitty/tree/master/more_autograding_examples
When the VM is “up”, you can connect from your host computer to the virtual machine via ssh. Windows users will need to install SSH software (e.g., WSL, or Cygwin, or Putty ). From a terminal in the repository directory type:
```
 vagrant ssh
```
You will connect to the VM as the root user.

If vagrant ssh asks for a password for the root@127.0.0.1 user and “vagrant” without the quotation marks does not work, look at the vagrant ssh config file and make note of the hostname and port.
```
 vagrant ssh-config
```
Then directly ssh into the VM by
```
 ssh vagrant@hostname -p port
```
If it asks for password, it should be “vagrant” and then
```
 sudo su
```
to login as the root user. You should then see you are logged in as root@vagrant.

The following users exist on the VM:

user	password	role
superuser	superuser	Superuser
vagrant	vagrant	OS user
root	vagrant	OS user
submitty_cgi	submitty_cgi	Submitty process
submitty_php	submitty_php	Submitty process
submitty_daemon	submitty_daemon	Submitty process
postgres	postgres	database process
instructor	instructor	Instructor submitty user
ta	ta	Full access grader submitty user
grader	grader	Limited access grader submitty user
student	student	Student submitty user

The VM has the following four courses by default and they are all part of the current semester:
- tutorial
- sample
- development
- blank
Note: The current semester is calculated by either using an s if in the month is < 7 else use f and then take the last two digits of the current year. So April 2017 would be s17 while September 2017 would be f17.

Starting and Stopping the Submitty VM

When you take a break from Submitty development work, you can suspend the Submitty VM to to save resources (CPU and battery) on your host machine.
```
vagrant suspend
```
Alternatively, you can halt the virtual machine. This is a more complete shutdown and will take slightly longer to restart when you resume development work.
```
vagrant halt
```
To resume work on a VM that is suspended or halted:
```
vagrant up
```
NOTE: when resuming work, you may see this warning several times, default: Warning: Remote connection disconnect. Retrying.. . These warnings are not harmful and can be ignored.
If you just want to restart the VM (same as halt then up), type:
```
vagrant reload
```
Read the Development Instructions page for instructions on updating an existing installation with recent code changes.
To completely delete the virtual machine, type:
```
vagrant destroy
```
And if desired (to start over from scratch with a fresh VM):
```
vagrant up
```

Installation Troubleshooting

If your install becomes stuck on SSH auth method: private key for more than three minutes, then what you can do is open Virtual Box, navigate to your VM, and click the green “Show” button. Having the VM’s terminal GUI open can seem to keep it awake for communication. While developing on Submitty, this “Show” terminal option can sometimes lead to the VM being faster with loads, so it may be worth having the GUI open even after your vagrant up.
If your install seems to randomly freeze during vagrant up with no explanation, then there are a couple of things that may be going wrong:

Note: If this happens, it may be worth running a simple vagrant destroy before you try to vagrant up again. It is possible to repair, but it is usually more effort than it is worth.
- First, check to make sure you have a solid internet connection. Even if the connection is fast, it may experience drop-outs every once in a while, so it is highly recommended to install it with a wired connection to the internet.
- Secondly, check to make sure your computer is not going to sleep, this can be changed in settings for some systems, or you can install a program to keep it awake (for example, amphetamine on Mac).
If an error is thrown during vagrant up, you may need to uninstall Virtual Box and all virtual machines by typing the following commands:

CAUTION: This should only be done if you do not have any other virtual machines.

To remove Virtual Box type:
```
 sudo apt-get remove --purge virtualbox
```
To remove all virtual machines and configuration files type:
```
 sudo rm ~/"VirtualBox VMs" -Rf
 sudo rm ~/.config/VirtualBox/ -Rf
```
This will delete all virtual machine settings. Then install the latest version of Virtual Box and vagrant from the links given in step 3 (using Ubuntu Software).
If it has been a while since your last vagrant destroy and vagrant up you may need to update/upgrade/reinstall the virtual box, vagrant, and the installed boxes on your system:

For example, on Mac:
```
  brew reinstall --cask virtualbox
  brew reinstall --cask vagrant
  vagrant plugin update
  vagrant box update
```
If you continue to have errors on Mac with vagrant up after reinstalling virtualbox and vagrant, check “System Preferences” -> “Security & Privacy”. You may need to approve/reapprove “Allow apps download from”. You may also need to restart your computer.

Similar instructions for other OS.
If this error is thrown during vagrant up command:
```
 ubuntu-26.84: E: Failed to fetch http://us.archive.ubuntu.com/ubuntu/pool/main/g/gcc-9gcc-9_9.4.0-1ubuntu1-20.84.1_amd64.deb Connection fail [IP: 189.211.241.193]
```
then you should use VPN to connect to the internet and then try vagrant up.This error usually occurs when you are outside the United States and the connection to the Ubuntu package repository is being blocked or restricted.

Note: The IP address provided in the error message may vary. ProtonVPN, which provides free VPN service can be used. You can download from their official website at (https://protonvpn.com). You can also use other VPN services.
See also Development Instructions Troubleshooting

Testing with a remote device

Make sure that the VM is stopped.
```
 vagrant halt
```
In the Vagrantfile, add a new port under config.vm.define for the primary box below the other forwarded ports (site, websockets, database).
```
ubuntu.vm.network 'forwarded_port', guest: 22, host: <port>, id: "ssh", host_ip: "0.0.0.0"
```
Replace <port> with the port you want to expose externally on the machine that is running the VM, and expose the specified port on the machine if necessary.
Start the vagrant machine with vagrant up.
Retrieve the private key for the vagrant machine, located at <SUBMITTY GIT REPO>/.vagrant/machines/<VM>/<VM BACKEND>/private_key.

At the time of writing, <VM> is ubuntu-22.04, and <VM BACKEND> is virtualbox.
Use SSH to connect from the remote device to the machine that is running the VM, and use SSH port forwarding (local forwarding) to forward the necessary ports.

The username to sign in is root and the authentication method is with private key (using the private key specified in part 4). If you encounter authentication issues, try adding vagrant as the password in addition to the private key.

For most things, you will only need to forward the site port and the websockets port (1511, 8443).

The configuration to set up the connection will differ based on your client; below is an example for if you’re using an SSH binary to connect, assuming that the SSH configuration file has the username, private key (identity file), IP address (host name), and port stored.
```
ssh -L 1511:localhost:1511 -L 8443:localhost:8443 $SUBMITTY_HOST
```
where $SUBMITTY_HOST is a reference to the Host from the SSH config file.

NOTE Especially for mobile operating systems, make sure that your SSH client supports SSH port forwarding. On iOS, you will also have to enable location tracking for the client to keep the connection alive in the background.

This has been tested with Blink for iOS and Termius for iOS (also available on Android, untested).
Navigate to localhost:1511 on the remote device.

Developer > Getting Started