Scope-harness is a high-level testing framework for scopes that offers high-level abstractions to interact with scopes and simulate user interactions in order to verify data (categories, results, departments etc.) returned by the scope. It can be used to implement tests executed as a part of the build process of a scope.
Scope-harness is available via C++ API and also offers bindings for Python 3. Both C++ and Python APIs offer same functionality. This documentation covers Python API only.
Scope harness for Python is build upon the standard unittest framework (by inheriting from ScopeHarnessTestCase, based on unittest.TestCase), but there no obligation to use it - the only functionality that ScopeHarnessTestCase provides is a helper assertMatchResult method, that can easily be replaced with a custom implementation.
Here is the implementation of assertMatchResult for your reference.
from unittest import TestCase
from scope_harness import MatchResult
class ScopeHarnessTestCase(TestCase):
""" A class whose instances are single test cases.
This class extends unittest.TestCase with helper methods relevant for testing of Unity scopes.
"""
def assertMatchResult(self, match_result):
""" Assert for MatchResult object that fails if match wasn't successful and prints
conditions which were not met by the matcher.
"""
self.assertIsInstance(match_result, MatchResult, msg='match_result must be an instance of MatchResult')
self.assertTrue(match_result.success, msg=match_result.concat_failures)
The main “entry point” for every scope harness test cases is an instance of ScopeHarness object. This object encapsulates various aspects of configuration of scopes runtime, including an instance of scoperegistry - the central process which maintains the list of known scopes, separate from the scoperegistry instance and scopes normally installed on your system.
When creating this object via one of its factory methods, you have to decide whether you want to run your tests against scoperegistry and scopes already installed on the system (see new_from_system()), scopesregistry executed against an existing configuration file (see new_from_pre_existing_config()) or a custom scope registry instance which only knows about scopes provided by your test (new_from_scope_list()). The latter is the most common use case.
Consider the following example of test setUpClass method which assumes two “dummy” scopes have been installed into your test directory, and TEST_DATA_DIR points to it.
from scope_harness import *
from scope_harness.testing import ScopeHarnessTestCase
import unittest
class MyTest(ScopeHarnessTestCase):
@classmethod
def setUpClass(cls):
cls.harness = ScopeHarness.new_from_scope_list(
Parameters([
TEST_DATA_DIR + "/myscope1/myscope1.ini",
TEST_DATA_DIR + "/myscope2/myscope2.ini"
])
)
Once ScopeHarness instance has been created, it provides the results_view property (a ResultsView instance) which corresponds to a scope page in the unity8 dash; you can set curently active scope, its current search query, change active department, inspect the returned categories and their results etc.
Consider the following simple test:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | class MyTest(ScopeHarnessTestCase):
@classmethod
def setUpClass(cls):
cls.harness = ScopeHarness.new_from_scope_list(Parameters([
TEST_DATA_DIR + "/myscope1/myscope1.ini"
]))
cls.view = cls.harness.results_view
def test_basic_result(self):
self.view.active_scope = 'myscope1'
self.view.search_query = ''
self.assertMatchResult(
CategoryListMatcher()
.has_at_least(2)
.mode(CategoryListMatcherMode.BY_ID)
.category(
CategoryMatcher('mycategory1')
.has_at_least(5)
.mode(CategoryMatcherMode.BY_URI)
.result(
ResultMatcher("myuri")
.properties({'title': 'mytitle', 'art':'myart'})
.dnd_uri("test:dnd_uri")
)
).match(self.view.categories)
)
|
4-6 - create main ScopeHarness scope harness object to interact with scope(s).
7 - store a reference to ResultsView object in the test case instance to reduce typing later.
10 - Make ‘myscope1’ the active scope.
11 - set search query value (executes a background search query).
check that there are at least 2 categories in the view (lines 13-14);
pick a specific category by its ID (15-17) and check that it has at least 5 results (line 18);
“dnd_uri” properties (lines 20-23).
whether the list contains at least N categories, or exactly N categories: use has_at_least() or has_exactly(), respectively.
whether the category has at least N results: use has_at_least().
def test_results(self):
self.view.search_query = ''
self.assertMatchResult(
CategoryListMatcher()
.has_at_least(5)
.mode(CategoryListMatcherMode.BY_ID)
.category(CategoryMatcher("app-of-the-week")
.has_at_least(1)
)
.category(CategoryMatcher("top-apps")
.has_at_least(1)
.mode(CategoryMatcherMode.STARTS_WITH)
.result(ResultMatcher("https://search.apps.ubuntu.com/api/v1/package/com.ubuntu.developer.bobo1993324.udropcabin")
.title('uDropCabin')
.subtitle('Zhang Boren')
))
.category(CategoryMatcher("our-favorite-games")
.has_at_least(1)
.mode(CategoryMatcherMode.BY_URI)
.result(ResultMatcher("https://search.apps.ubuntu.com/api/v1/package/com.ubuntu.developer.andrew-hayzen.volleyball2d") \
))
.category(CategoryMatcher("travel-apps")
.has_at_least(1))
.match(self.view.categories))
Departments can be “browsed” by calling browse_department() method; changing the department invokes a new search and the method returns the new list of departments. The list of departments can be tested using DepartmentMatcher and ChildDepartmentMatcher matchers. The DepartmentMatcher support three modes of matching (DepartmentMatcherMode.ALL, DepartmentMatcherMode.STARTS_WITH and DepartmentMatcherMode.BY_ID) which have the same semantics as with CategoryMatcher or CategoryListMatcher described above.
Here is an example of two departments tests: the first test case checks the starting list of departments (the surfacing mode), the second case simulates browsing of games sub-department, verifies it has no further sub-departments and also verifies the returned categories.
Note: the empty department ID corresponds to the root department.
def test_surfacing_departments(self):
self.view.search_query = ''
departments = self.view.browse_department('')
self.assertMatchResult(
DepartmentMatcher()
.mode(DepartmentMatcherMode.STARTS_WITH)
.id('')
.label('All')
.all_label('')
.parent_id('')
.parent_label('')
.is_root(True)
.is_hidden(False)
.child(ChildDepartmentMatcher('business'))
.child(ChildDepartmentMatcher('communication'))
.child(ChildDepartmentMatcher('education'))
.child(ChildDepartmentMatcher('entertainment'))
.child(ChildDepartmentMatcher('finance'))
.child(ChildDepartmentMatcher('games'))
.child(ChildDepartmentMatcher('graphics'))
.child(ChildDepartmentMatcher('accessories'))
.child(ChildDepartmentMatcher('weather'))
.match(departments))
def test_department_browsing(self):
self.view.search_query = ''
departments = self.view.browse_department('games')
self.assertMatchResult(DepartmentMatcher()
.has_exactly(0)
.mode(DepartmentMatcherMode.STARTS_WITH)
.label('Games')
.all_label('')
.parent_id('')
.parent_label('All')
.is_root(False)
.is_hidden(False)
.match(departments))
self.assertMatchResult(CategoryListMatcher()
.has_exactly(3)
.mode(CategoryListMatcherMode.BY_ID)
.category(CategoryMatcher("top-games")
.has_at_least(1)
)
.category(CategoryMatcher("all-scopes")
.has_at_least(1)
)
.category(CategoryMatcher("all-apps")
.has_at_least(1)
)
.match(self.view.categories))
Previews can be invoked by calling tap() method of the result. Note that tapping the result will - in cases where result’s uri is a canned scope query (i.e. scope:// uri) - execute a new search and return a ResultsView instance; in other cases a PreviewView will be returned. This conditions are verified by checks in lines 5 and 37.
Below is an example of test cases covering preview widgets. The test_preview_layouts test case verifies different column layouts within the preview. The second test case simulates activation of preview action by calling trigger() (line 47) and verifies the same preview is returned in response.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | def test_preview_layouts(self):
self.view.search_query = ''
pview = self.view.category(0).result(0).tap()
self.assertIsInstance(pview, PreviewView)
self.assertMatchResult(PreviewColumnMatcher().column(
PreviewMatcher()
.widget(PreviewWidgetMatcher("img"))
.widget(PreviewWidgetMatcher("hdr"))
.widget(PreviewWidgetMatcher("desc"))
.widget(PreviewWidgetMatcher("actions"))
).match(pview.widgets))
pview.column_count = 2
self.assertMatchResult(PreviewColumnMatcher()
.column(PreviewMatcher()
.widget(PreviewWidgetMatcher("img")))
.column(PreviewMatcher()
.widget(PreviewWidgetMatcher("hdr"))
.widget(PreviewWidgetMatcher("desc"))
.widget(PreviewWidgetMatcher("actions"))
).match(pview.widgets))
pview.column_count = 1
self.assertMatchResult(PreviewColumnMatcher()
.column(PreviewMatcher()
.widget(PreviewWidgetMatcher("img"))
.widget(PreviewWidgetMatcher("hdr"))
.widget(PreviewWidgetMatcher("desc"))
.widget(PreviewWidgetMatcher("actions"))
).match(pview.widgets))
def test_preview_action(self):
self.view.search_query = ''
pview = self.view.category(0).result(0).tap()
self.assertIsInstance(pview, PreviewView)
self.assertMatchResult(PreviewColumnMatcher()
.column(PreviewMatcher()
.widget(PreviewWidgetMatcher("img"))
.widget(PreviewWidgetMatcher("hdr"))
.widget(PreviewWidgetMatcher("desc"))
.widget(PreviewWidgetMatcher("actions"))
).match(pview.widgets))
next_view = pview.widgets_in_first_column["actions"].trigger("hide", None)
self.assertEqual(pview, next_view)
|
Settings exported by scopes can be accessed via settings() property and tested using SettingsMatcher. The SettingsView object returned by the above method set() method that can be used to modify settings (simulate user choices). Note that set method is loosely-typed (the new value is an object / variant), that means the correct data type needs to be passed to it, depending on the type of setting to modify:
- for a setting of number type, pass an integer or float number.
- for a setting of string type, pass a string value.
- for a setting of list type, pass the string value corresponding to one of the supported choices.
- for a setting of boolean type, pass True / False literals.
Changing a setting value refreshes search results.
Here is an example of a test case which modifies a setting value (this test should of course also check the new results after settings change; omitted here).
def test_settings_change(self):
self.view.active_scope = 'mock-scope'
settings = self.view.settings
settings.set("location", "Barcelona")
self.assertMatchResult(
SettingsMatcher()
.mode(SettingsMatcherMode.BY_ID)
.option(
SettingsOptionMatcher("location")
.value("Barcelona")
)
.match(settings)
)