Racing Capybara, Puma and Chrome for Testing

Write More Reliable Browser Test

@Ruby Meetup Frankfurt

 

 

Author: Konstantin Filtschew
Presentation: 2020-01-30
Last Update: 2020-02-03

Introduction

Agenda

  • rack-test
  • Browser Tests
  • Examples
  • Help finding flaky tests
  • Browser tests on CI/CD
  • Exchange of knowledge

rack-test

            
  visit root_url
  # capybara: load requested url

  # rack-test: fake Rack request
  # rack-test: in-memory instance of Rails
  # rack-test: response parse by rack-test
  # rack-test: saved as current page

  # capybara: ...

rack-test

  • Good
    • Small
    • Easy
    • Reliable
  • Bad
    • No Javascript

Browser Tests

Running a test using Selenium, capybara-webkit, Poltergeist, ...

  • Process forking
  • Background threads
  • HTTP requests

Understand:

  • running
  • race conditions

Capybara to the Rescue

Capybara to the Rescue

Capybara to the Rescue

Capybara to the Rescue

Capybara to the Rescue

Methods that do wait

  • find, find_field, find_link and other finders
  • within
  • has_selector?, has_no_selector? and similar
  • click_link, fill_in, check, select, choose and other actions

 

Do also wait since Capybara 3.0rc1:

  • all(selector)
  • first(selector)

Methods that don't wait

  • visit(path)
  • current_path
  • execute_script, evaluate_script
  • simple accessors: text, value, title, etc

First matching element

Bad

            
  first(".active").click

Good

              
 find(".active").click

All matching

Bad

            
  all(".active").each(&:click)

Good

              
  find(".active", match: :first)
  all(".active").each(&:click)

Directly Javascript

Bad

            
  execute_script("$('.active').focus()")

Good

              
  find(".active")
  execute_script("$('.active').focus()")

Checking a field’s value

Bad

            
  expect(find_field("Name").value).to eq("Kevin")

Good

              
  expect(page).to have_field("Name", with: "Kevin")

Looking for matching CSS

Bad

            
  expect(has_css?(".active")).to be_falsey

Good

              
  expect(page).not_to have_css(".active")

Something not there

Bad

            
  expect(page).to !have_css(".some-class")

Good

              
  expect(page).to have_no_css(".some-class")

Something not there

Good

            
  expect(page).to have_no_css(".some-class")

Better

              
  expect(page).to_not have_css(".some-class")

Don't mix things up

  • Think and test like a user
  • Users don’t lookup databases
    • They have no access
    • They don’t even know there is one
  • Don't test code flow
    • Test workflows
    • Don't lookup details
    • Separation of concerns

Finding flaky tests

  • You'll have an eye for this.
  • Grep for examples:
    • first(
    • expect(find
    • expect(page).to !
    • the one you have found

 

Still won’t find them all

Still fail randomly

            
  Capybara.default_max_wait_time
  • default: 2 seconds
  • increase?
  • decrease?

Get some sleep?

            
  sleep 1

Wait, can we use it to fail flaky tests?

              
  # app/controllers/application_controller.rb
  after_action -> { sleep(0.5) }

Ideas for slower actions

            
  # get extra time
  expect(page).to have_current_path(expected_path)
  expect(page).to have_content("page_title")
  expect(page).to have_content("element_below")

 

Look up things that should already be there, buy additional time.

Summary

  • Feature tests are different
  • Think like a user
  • Wait like a user until X is finished/loaded/enabled
  • Don't sleep 😜
  • Check reference configuration

Read carefully and understand

Running on CI/CD

Understand the underlying architecture

  • Cores
  • RAM
  • Storage
  • Network

Thank you

https://bit.ly/2RsxDfM