Skip to content

Homework 4: Software Testing

Learning goals

  • Get comfortable with unit testing frameworks
  • Learn how to create continuous integration workflows
  • Create test infrastructure and code for the calculator project

Project context

You and your team have access to a basic calculator app through GitHub Classroom. Last week, you learned how to set up the repository and merge in some pull requests (PRs). This week, you will learn to write test cases and create a continuous integration (CI) pipeline. You should apply the development processes you learn here to your prototype product.

Assignment instructions

In this assignment, you'll go to your local development environment for the calculator app, install the testing framework, create some test cases, run them, and then set up a CI workflow to run these every time new code is checked into your repository.

Unit testing

Unit testing is the lowest level manner of testing your software. In unit testing, we write test cases to exercise every individual function in our software. Each unit test exercises just one program path in (usually) just one function. It takes a lot of unit tests to cover all the code paths and ensure that your functions work as intended.

We can use a unit testing framework to make it easier to run these test cases and automatically tally the results.

1. Configure Jest

We will be using the Jest unit testing framework for Javascript applications running in Node.js and in the browser.

Follow these instructions to install the Jest framework in your calculator front end and backend applications. We have already created a test directory in calculator-backend to store your new test cases. The tests in the frontend code are in the src/ subdirectory of calculator-frontend.

Let's fix one configuration error in the calculator code; this is necessary because of a bug in Jest/Axios interactions.

Edit calculator-frontend/package.json. Change the line starting with "test" inside of the "scripts" object to read:

"test": "react-scripts test --transformIgnorePatterns 'node_modules/(?!(axios)/)' --detectOpenHandles",

Turn in instructions

There is no deliverable for this step.

2. Run our test cases

Jest runs all files whose names end in .test.js. In calculator-backend/tests, there's a file called calculate.test.js. The code currently in there tests the calculate function by evaluating 2+3 and making sure the function returns 5.

Run the test by executing:

cd calculator-backend
npm test

It should execute the test and show that it passed.

Turn in instructions

There is no deliverable for this step.

3. Write your first test cases

Let's add some test cases to calculator-backend.

  1. Add test cases for the '-', '*', and '/' operators that look like the test case for '+'.
  2. For division, test what happens if someone tries to divide by 0.
  3. Run these test cases by running 'npm test'. Do the test cases pass?

Turn in instructions

Take a screenshot that shows your new test cases passing and put this in the PDF you turn in.

3. Write a test case for the front end

You'll need to install more npm modules to test the front end.

Execute these three commands:

npm install @testing-library/react
npm install @testing-library/jest-dom

There's already an App.test.js in the src/ directory. It has the test code for App.js.

Let's run that using npm test.

This looks like a bad test. If you open up App.test.js it creates the Calculator app React element and then looks for a box on the screen labeled 'learn react'.

That's not right! Our calculator is called 'Calculator App'. Change the test case to look for the text 'Calculator App' instead of 'learn react'. That's better! Be sure to update the test text 'renders calculator app react element' so the person running the test cases knows what the test does.

If something went wrong in the test code, you can fix it, save the file, and 'npm test' will reexecute all the test cases without having to start it over again.

Turn in instructions

Take a screenshot that shows your modified test case passing and put this in the PDF you turn in.

4. Write more test cases for the front end

We want to test the calculator display to see if pressing the number buttons properly shows the numbers.

First, we need to add an HTML attribute to the appropriate div so we can refer to it in our test cases.

In Calculator.js, add the attribute 'data-testid="total"' to the div with the className 'expression'. It should look like this:

<div className="expression" data-testid ="total">{expression || "0"}</div>

Second, in the calculator-frontend/src directory, add a new file called Calculate.tests.js. Put this code in your file:

import { render, fireEvent, screen } from '@testing-library/react'
import '@testing-library/jest-dom'
import Calculator from './Calculator'

it('Appends the number when pressed', async () => {
   render(<Calculator/>);

   // press a 1 on the calculator
   var button1 = screen.getByText('1')
   fireEvent.click(button1)
   await button1

   // get the expression DOM node and check that it is 1
   const element = screen.getByTestId('total')
   expect(element).toHaveTextContent('1');

   // press a 2 on the calculator
   var button2 = screen.getByText('2')
   fireEvent.click(button2)
   await button2

   // get the expression DOM node and check that it is 12
   expect(element).toHaveTextContent('12');
});

It checks that if you press a '1', you see a 1 in the top of the calculator. If you then press a '2', you should see a 12 at the top of the calculator.

Does it work?

Let's add some test cases to calculator-frontend.

  1. Add a test case for the 'C' button. It should clear the calculator display back to 0.
  2. Add a test case for the '=' button. It should read the expression in the display, calculate the right answer, and check that the backend returned that answer to be displayed.
  3. Run these test cases by running 'npm test'. Do the test cases pass?

Turn in instructions

Take a screenshot that shows your new test cases passing and put this in the PDF you turn in.

5. Test your new calculator functionality

Each team member should choose a feature added to the calculator in HW3 that they did not write. Coordinate with your teammates to ensure that no two members test the same feature.

  1. Create up to 5 new test cases for the selected feature. Some will be testable with just 1, but others may require more tests to cover all of the possible functionality.

Turn in instructions

Take a screenshot that shows your new test cases passing and put this in your PDF.

6. Check in your test code.

Create a PR to commit your test code to the team's repository. Use your PR message to clearly describe the tests you've written.

Turn in instructions

Please provide us a link to your PR.

7. Review at least two PRs

Following the best practices we discussed in class, leave two reviews on your teammates' PRs. If there issues, continue to discuss in the comments until you're satisfied with the changes and able to approve the pull requests.

Turn in instructions

Please provide two or more links to your reviews.

10. Merge your reviewed PR

Finally, once your PR has been approved, merge it.

Turn in instructions

Please provide a link to your merged commit.

8. Create a continuous integration pipeline to run your test cases

This is a group exercise. Only one person should PR the workflow file, and others should review.

To ensure the quality of our codebase, we want to make sure that your code passes all its test cases after every checkin. If the tests fail, we should reject the checkin.

To setup our CI workflow, first, create a workflows subdirectory in .github. Then create a new file called ci.yml in workflows/.

Open ci.yml in a text editor, paste in the following code snippet, and commit this to your repository. Use hygienic code practices and commit to a branch first, then issue a pull request, do a code review, and merge it into main.

name: ci
on:
  push:
    branches:
      - main
  pull_request:
    branches: [main]

permissions:
  contents: write

jobs:
  jest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run Frontend Tests
        working-directory: ./calculator-frontend
        run: |
          npm install
          npm test
      - name: Run Backend Tests
        working-directory: ./calculator-backend
        run: |
          npm install
          npm test

Once this is installed, every time we push to main or do a pull request to main, this will execute the frontend and backend test code. We have Require status checks to pass before merging turned on in our repository settings. If any tests fail, the push or pull request will fail as well.

You can find out what happened in the tests by clicking on the Actions tab at the top of your GitHub repository page and choose the 'ci' workflow. If the tests worked, you'll see all green dots. If it's running right now, the dot will be yellow and show a spinning circle. If there was a failure, the dot will be red. Click on a dot to see a terminal window that shows the commands as they executed as well as their results.

Turn in instructions

Turn in the URL to your workflow run of ci.yml that resulted in a successful test case run. If you had to run the ci workflow several times before it passed, turn in the URL of the latest workflow run that succeeded.