Integration Testing

In addition to unit tests, we run integration tests using ChromeDriver and WebdriverIO through the Spectron library.

Running Tests

script/grunt run-integration-tests

This command will, in order:

1. Run npm test in the /spec_integration directory and pass in the NYLAS_ROOT_PATH
2. Boot jasmine and load all files ending in -spec
3. Most tests in beforeAll will boot Mailspring via the MailspringLauncher. See spec_integration/helpers/Mailspring-launcher.es6
4. This instantiates a new spectron Application which will spawn a ChromeDriver process with the appropriate Mailspring launch args.
5. ChromeDriver will then boot a Selenium server at http://localhost:9515
6. The ChromeDriver / Selenium server will boot Mailspring with testing hooks and expose an controlling API.
7. The API is made easily available through the Spectron API
8. The MailspringLauncher's mainWindowReady or popoutComposerWindowReady or onboardingWindowReady methods poll the app until the designated window is available and loaded. Then will resolve a Promise once everything has booted.

Writing Tests

The Spectron API is a pure extension over the Webdriver API. Reference both to write tests.

Most of the methods on app.client object apply to the "currently focused" window only. Since Mailspring has several windows (many of which are hidden) the MailspringLauncher extension will cycle through windows automatically until it finds the one you want, and then select it.

Furthermore, "loaded" in the pure Spectron sense is only once the window is booted. Mailspring windows take much longer to full finish loading packages and rendering the UI. The MailspringLauncher::windowReady method and its derivatives take this into account.

You will almost always need the minimal boilerplate for each integration test:

describe('My test', () => {
  beforeAll((done)=>{
    // Boot in dev mode with no arguments
    this.app = new MailspringLauncher(["--dev"]);
    this.app.mainWindowReady().finally(done);
  });

  afterAll((done)=> {
    if (this.app && this.app.isRunning()) {
      this.app.stop().finally(done);
    } else {
      done()
    }
  });

  it("is a test you'll write", () => {
  });

  it("is an async test you'll write", (done) => {
    doSomething.finally(done)
  });
});

Executing Code in Mailspring's environment

The app.client.execute and app.client.executeAsync methods are extremely helpful when running code in Mailspring. Those are documented slightly more on the WebdriveIO API docs page here

it("is a test you'll write", () => {
  this.app.client.execute((arg1)=>{
    // NOTE: `arg1` just got passed in over a JSON api. It can only be a
    // primitive data type

    // I'M RUNNING IN Mailspring
    return someValue

  }, arg1).then(({value})=>{
    // NOTE: the return is stuffed in an attribute called `value`. Also it
    // passed back of a JSON API and can only be a primitive value.
  })
});

Debugging tests.

Debugging is through lots of console.loging.

There is code is spec_integration/jasmine/bootstrap.js that attempts to catch unhandled Promises and color them accordingly.

If you want to access logs from within Mailspring via the app.client.execute blocks, you'll have to either package it up yourself and return it, or use the new app.client.getMainProcessLogs() just added into Spectron.