mobly package

Subpackages

Submodules

mobly.asserts module

mobly.asserts.abort_all(reason, extras=None)[source]

Abort all subsequent tests, including the ones not in this test class or iteration.

Parameters:
  • reason – The reason to abort.
  • extras – An optional field for extra information to be included in test result.
Raises:

signals.TestAbortAll – Abort all subsequent tests.

mobly.asserts.abort_all_if(expr, reason, extras=None)[source]

Abort all subsequent tests, if the expression evaluates to True.

Parameters:
  • expr – The expression that is evaluated.
  • reason – The reason to abort.
  • extras – An optional field for extra information to be included in test result.
Raises:

signals.TestAbortAll – Abort all subsequent tests.

mobly.asserts.abort_class(reason, extras=None)[source]

Abort all subsequent tests within the same test class in one iteration.

If one test class is requested multiple times in a test run, this can only abort one of the requested executions, NOT all.

Parameters:
  • reason – The reason to abort.
  • extras – An optional field for extra information to be included in test result.
Raises:

signals.TestAbortClass – Abort all subsequent tests in a test class.

mobly.asserts.abort_class_if(expr, reason, extras=None)[source]

Abort all subsequent tests within the same test class in one iteration, if expression evaluates to True.

If one test class is requested multiple times in a test run, this can only abort one of the requested executions, NOT all.

Parameters:
  • expr – The expression that is evaluated.
  • reason – The reason to abort.
  • extras – An optional field for extra information to be included in test result.
Raises:

signals.TestAbortClass – Abort all subsequent tests in a test class.

mobly.asserts.assert_equal(first, second, msg=None, extras=None)[source]

Assert the equality of objects, otherwise fail the test.

Error message is “first != second” by default. Additional explanation can be supplied in the message.

Parameters:
  • first – The first object to compare.
  • second – The second object to compare.
  • msg – A string that adds additional info about the failure.
  • extras – An optional field for extra information to be included in test result.
mobly.asserts.assert_false(expr, msg, extras=None)[source]

Assert an expression evaluates to false, otherwise fail the test.

Parameters:
  • expr – The expression that is evaluated.
  • msg – A string explaining the details in case of failure.
  • extras – An optional field for extra information to be included in test result.
mobly.asserts.assert_raises(expected_exception, extras=None, *args, **kwargs)[source]

Assert that an exception is raised when a function is called.

If no exception is raised, test fail. If an exception is raised but not of the expected type, the exception is let through.

This should only be used as a context manager:
with assert_raises(Exception):
func()
Parameters:
  • expected_exception – An exception class that is expected to be raised.
  • extras – An optional field for extra information to be included in test result.
mobly.asserts.assert_raises_regex(expected_exception, expected_regex, extras=None, *args, **kwargs)[source]

Assert that an exception is raised when a function is called.

If no exception is raised, test fail. If an exception is raised but not of the expected type, the exception is let through. If an exception of the expected type is raised but the error message does not match the expected_regex, test fail.

This should only be used as a context manager:
with assert_raises(Exception):
func()
Parameters:
  • expected_exception – An exception class that is expected to be raised.
  • extras – An optional field for extra information to be included in test result.
mobly.asserts.assert_true(expr, msg, extras=None)[source]

Assert an expression evaluates to true, otherwise fail the test.

Parameters:
  • expr – The expression that is evaluated.
  • msg – A string explaining the details in case of failure.
  • extras – An optional field for extra information to be included in test result.
mobly.asserts.explicit_pass(msg, extras=None)[source]

Explicitly pass a test.

This will pass the test explicitly regardless of any other error happened in the test body. E.g. even if errors have been recorded with expects, the test will still be marked pass if this is called.

A test without uncaught exception will pass implicitly so this should be used scarcely.

Parameters:
  • msg – A string explaining the details of the passed test.
  • extras – An optional field for extra information to be included in test result.
Raises:

signals.TestPass – Mark a test as passed.

mobly.asserts.fail(msg, extras=None)[source]

Explicitly fail a test.

Parameters:
  • msg – A string explaining the details of the failure.
  • extras – An optional field for extra information to be included in test result.
Raises:

signals.TestFailure – Mark a test as failed.

mobly.asserts.skip(reason, extras=None)[source]

Skip a test.

Parameters:
  • reason – The reason this test is skipped.
  • extras – An optional field for extra information to be included in test result.
Raises:

signals.TestSkip – Mark a test as skipped.

mobly.asserts.skip_if(expr, reason, extras=None)[source]

Skip a test if expression evaluates to True.

Parameters:
  • expr – The expression that is evaluated.
  • reason – The reason this test is skipped.
  • extras – An optional field for extra information to be included in test result.

mobly.base_instrumentation_test module

class mobly.base_instrumentation_test.BaseInstrumentationTestClass(configs)[source]

Bases: mobly.base_instrumentation_test.InstrumentationTestMixin, mobly.base_test.BaseTestClass

Base class for all instrumentation test classes to inherit from.

This class extends the BaseTestClass to add functionality to run and parse the output of instrumentation runs.

DEFAULT_INSTRUMENTATION_OPTION_PREFIX

string, the default prefix for instrumentation params contained within user params.

DEFAULT_INSTRUMENTATION_ERROR_MESSAGE

string, the default error message to set if something has prevented something in the instrumentation test run from completing properly.

class mobly.base_instrumentation_test.InstrumentationTestMixin[source]

Bases: object

A mixin for Mobly test classes to inherit from for instrumentation tests.

This class should be used in a subclass of both BaseTestClass and this class in order to provide instrumentation test capabilities. This mixin is explicitly for the case where the underlying BaseTestClass cannot be replaced with BaseInstrumentationTestClass. In general, prefer using BaseInstrumentationTestClass instead.

DEFAULT_INSTRUMENTATION_OPTION_PREFIX

string, the default prefix for instrumentation params contained within user params.

DEFAULT_INSTRUMENTATION_ERROR_MESSAGE

string, the default error message to set if something has prevented something in the instrumentation test run from completing properly.

DEFAULT_INSTRUMENTATION_ERROR_MESSAGE = 'instrumentation run exited unexpectedly'
DEFAULT_INSTRUMENTATION_OPTION_PREFIX = 'instrumentation_option_'
parse_instrumentation_options(parameters=None)[source]

Returns the options for the instrumentation test from user_params.

By default, this method assume that the correct instrumentation options all start with DEFAULT_INSTRUMENTATION_OPTION_PREFIX.

Parameters:parameters – dict, the key value pairs representing an assortment of parameters including instrumentation options. Usually, this argument will be from self.user_params.
Returns:A dictionary of options/parameters for the instrumentation tst.
run_instrumentation_test(device, package, options=None, prefix=None, runner=None)[source]

Runs instrumentation tests on a device and creates test records.

Parameters:
  • device – AndroidDevice, the device to run instrumentation tests on.
  • package – string, the package name of the instrumentation tests.
  • options – dict, Instrumentation options for the instrumentation tests.
  • prefix – string, an optional prefix for parser output for distinguishing between instrumentation test runs.
  • runner – string, the runner to use for the instrumentation package, default to DEFAULT_INSTRUMENTATION_RUNNER.
Returns:

A boolean indicating whether or not all the instrumentation test

methods passed.

Raises:

TestError if the instrumentation run crashed or if parsing the – output failed.

mobly.base_test module

class mobly.base_test.BaseTestClass(configs)[source]

Bases: object

Base class for all test classes to inherit from.

This class gets all the controller objects from test_runner and executes the tests requested within itself.

Most attributes of this class are set at runtime based on the configuration provided.

The default logger in logging module is set up for each test run. If you want to log info to the test run output file, use logging directly, like logging.info.

tests

A list of strings, each representing a test method name.

TAG

A string used to refer to a test class. Default is the test class name.

results

A records.TestResult object for aggregating test results from the execution of tests.

controller_configs

dict, controller configs provided by the user via test bed config.

current_test_info

RuntimeTestInfo, runtime information on the test currently being executed.

root_output_path

string, storage path for output files associated with the entire test run. A test run can have multiple test class executions. This includes the test summary and Mobly log files.

log_path

string, storage path for files specific to a single test class execution.

test_bed_name

[Deprecated, use ‘testbed_name’ instead] string, the name of the test bed used by a test run.

testbed_name

string, the name of the test bed used by a test run.

user_params

dict, custom parameters from user, to be consumed by the test logic.

TAG = None
exec_one_test(test_name, test_method)[source]

Executes one test and update test results.

Executes setup_test, the test method, and teardown_test; then creates a records.TestResultRecord object with the execution information and adds the record to the test class’s test results.

Parameters:
  • test_name – string, Name of the test.
  • test_method – function, The test method to execute.
generate_tests(test_logic, name_func, arg_sets, uid_func=None)[source]

Generates tests in the test class.

This function has to be called inside a test class’s self.setup_generated_tests function.

Generated tests are not written down as methods, but as a list of parameter sets. This way we reduce code repetition and improve test scalability.

Users can provide an optional function to specify the UID of each test. Not all generated tests are required to have UID.

Parameters:
  • test_logic – function, the common logic shared by all the generated tests.
  • name_func – function, generate a test name according to a set of test arguments. This function should take the same arguments as the test logic function.
  • arg_sets – a list of tuples, each tuple is a set of arguments to be passed to the test logic function and name function.
  • uid_func – function, an optional function that takes the same arguments as the test logic function and returns a string that is the corresponding UID.
get_existing_test_names()[source]

Gets the names of existing tests in the class.

A method in the class is considered a test if its name starts with ‘test_*’.

Note this only gets the names of tests that already exist. If setup_generated_test has not happened when this was called, the generated tests won’t be listed.

Returns:A list of strings, each is a test method name.
on_fail(record)[source]

A function that is executed upon a test failure.

User implementation is optional.

Parameters:record – records.TestResultRecord, a copy of the test record for this test, containing all information of the test execution including exception objects.
on_pass(record)[source]

A function that is executed upon a test passing.

Implementation is optional.

Parameters:record – records.TestResultRecord, a copy of the test record for this test, containing all information of the test execution including exception objects.
on_skip(record)[source]

A function that is executed upon a test being skipped.

Implementation is optional.

Parameters:record – records.TestResultRecord, a copy of the test record for this test, containing all information of the test execution including exception objects.
record_data(content)[source]

Record an entry in test summary file.

Sometimes additional data need to be recorded in summary file for debugging or post-test analysis.

Each call adds a new entry to the summary file, with no guarantee of its position among the summary file entries.

The content should be a dict. If absent, timestamp field is added for ease of parsing later.

Parameters:content – dict, the data to add to summary file.
register_controller(module, required=True, min_number=1)[source]

Loads a controller module and returns its loaded devices.

A Mobly controller module is a Python lib that can be used to control a device, service, or equipment. To be Mobly compatible, a controller module needs to have the following members:

def create(configs):
    [Required] Creates controller objects from configurations.

    Args:
        configs: A list of serialized data like string/dict. Each
            element of the list is a configuration for a controller
            object.

    Returns:
        A list of objects.

def destroy(objects):
    [Required] Destroys controller objects created by the create
    function. Each controller object shall be properly cleaned up
    and all the resources held should be released, e.g. memory
    allocation, sockets, file handlers etc.

    Args:
        A list of controller objects created by the create function.

def get_info(objects):
    [Optional] Gets info from the controller objects used in a test
    run. The info will be included in test_summary.yaml under
    the key 'ControllerInfo'. Such information could include unique
    ID, version, or anything that could be useful for describing the
    test bed and debugging.

    Args:
        objects: A list of controller objects created by the create
            function.

    Returns:
        A list of json serializable objects: each represents the
            info of a controller object. The order of the info
            object should follow that of the input objects.

Registering a controller module declares a test class’s dependency the controller. If the module config exists and the module matches the controller interface, controller objects will be instantiated with corresponding configs. The module should be imported first.

Parameters:
  • module – A module that follows the controller module interface.
  • required – A bool. If True, failing to register the specified controller module raises exceptions. If False, the objects failed to instantiate will be skipped.
  • min_number – An integer that is the minimum number of controller objects to be created. Default is one, since you should not register a controller module without expecting at least one object.
Returns:

A list of controller objects instantiated from controller_module, or None if no config existed for this controller and it was not a required controller.

Raises:

ControllerError – * The controller module has already been registered. * The actual number of objects instantiated is less than the * min_number. * required is True and no corresponding config can be found. * Any other error occurred in the registration process.

run(test_names=None)[source]

Runs tests within a test class.

One of these test method lists will be executed, shown here in priority order:

  1. The test_names list, which is passed from cmd line. Invalid names are guarded by cmd line arg parsing.
  2. The self.tests list defined in test class. Invalid names are ignored.
  3. All function that matches test method naming convention in the test class.
Parameters:test_names – A list of string that are test method names requested in cmd line.
Returns:The test results object of this class.
setup_class()[source]

Setup function that will be called before executing any test in the class.

To signal setup failure, use asserts or raise your own exception.

Errors raised from setup_class will trigger on_fail.

Implementation is optional.

setup_generated_tests()[source]

Preprocesses that need to be done before setup_class.

This phase is used to do pre-test processes like generating tests. This is the only place self.generate_tests should be called.

If this function throws an error, the test class will be marked failure and the “Requested” field will be 0 because the number of tests requested is unknown at this point.

setup_test()[source]

Setup function that will be called every time before executing each test method in the test class.

To signal setup failure, use asserts or raise your own exception.

Implementation is optional.

teardown_class()[source]

Teardown function that will be called after all the selected tests in the test class have been executed.

Errors raised from teardown_class do not trigger on_fail.

Implementation is optional.

teardown_test()[source]

Teardown function that will be called every time a test method has been executed.

Implementation is optional.

unpack_userparams(req_param_names=None, opt_param_names=None, **kwargs)[source]

An optional function that unpacks user defined parameters into individual variables.

After unpacking, the params can be directly accessed with self.xxx.

If a required param is not provided, an exception is raised. If an optional param is not provided, a warning line will be logged.

To provide a param, add it in the config file or pass it in as a kwarg. If a param appears in both the config file and kwarg, the value in the config file is used.

User params from the config file can also be directly accessed in self.user_params.

Parameters:
  • req_param_names – A list of names of the required user params.
  • opt_param_names – A list of names of the optional user params.
  • **kwargs – Arguments that provide default values. e.g. unpack_userparams(required_list, opt_list, arg_a=’hello’) self.arg_a will be ‘hello’ unless it is specified again in required_list or opt_list.
Raises:

Error – A required user params is not provided.

exception mobly.base_test.Error[source]

Bases: Exception

Raised for exceptions that occurred in BaseTestClass.

mobly.config_parser module

exception mobly.config_parser.MoblyConfigError[source]

Bases: Exception

Raised when there is a problem in test configuration file.

class mobly.config_parser.TestRunConfig[source]

Bases: object

The data class that holds all the information needed for a test run.

log_path

string, specifies the root directory for all logs written by a test run.

test_bed_name

[Deprecated, use ‘testbed_name’ instead] string, the name of the test bed used by a test run.

testbed_name

string, the name of the test bed used by a test run.

controller_configs

dict, configs used for instantiating controller objects.

user_params

dict, all the parameters to be consumed by the test logic.

summary_writer

records.TestSummaryWriter, used to write elements to the test result summary file.

test_class_name_suffix

string, suffix to append to the class name for reporting. This is used for differentiating the same class executed with different parameters in a suite.

copy()[source]

Returns a deep copy of the current config.

mobly.config_parser.load_test_config_file(test_config_path, tb_filters=None)[source]

Processes the test configuration file provied by user.

Loads the configuration file into a dict, unpacks each testbed config into its own dict, and validate the configuration in the process.

Parameters:
  • test_config_path – Path to the test configuration file.
  • tb_filters – A subset of test bed names to be pulled from the config file. If None, then all test beds will be selected.
Returns:

A list of test configuration dicts to be passed to test_runner.TestRunner.

mobly.controller_manager module

Module for Mobly controller management.

class mobly.controller_manager.ControllerManager(class_name, controller_configs)[source]

Bases: object

Manages the controller objects for Mobly tests.

This manages the life cycles and info retrieval of all controller objects used in a test.

controller_configs

dict, controller configs provided by the user via test bed config.

get_controller_info_records()[source]

Get the info records for all the controller objects in the manager.

New info records for each controller object are created for every call so the latest info is included.

Returns:List of records.ControllerInfoRecord objects. Each opject conatins the info of a type of controller
register_controller(module, required=True, min_number=1)[source]

Loads a controller module and returns its loaded devices.

This is to be used in a mobly test class.

Parameters:
  • module – A module that follows the controller module interface.
  • required – A bool. If True, failing to register the specified controller module raises exceptions. If False, the objects failed to instantiate will be skipped.
  • min_number – An integer that is the minimum number of controller objects to be created. Default is one, since you should not register a controller module without expecting at least one object.
Returns:

A list of controller objects instantiated from controller_module, or None if no config existed for this controller and it was not a required controller.

Raises:

ControllerError – * The controller module has already been registered. * The actual number of objects instantiated is less than the * min_number. * required is True and no corresponding config can be found. * Any other error occurred in the registration process.

unregister_controllers()[source]

Destroy controller objects and clear internal registry.

This will be called after each test class.

mobly.controller_manager.verify_controller_module(module)[source]

Verifies a module object follows the required interface for controllers.

The interface is explained in the docstring of base_test.BaseTestClass.register_controller.

Parameters:module – An object that is a controller module. This is usually imported with import statements or loaded by importlib.
Raises:ControllerError – if the module does not match the Mobly controller interface, or one of the required members is null.

mobly.expects module

mobly.expects.expect_equal(first, second, msg=None, extras=None)[source]

Expects the equality of objects, otherwise fail the test.

If the expectation is not met, the test is marked as fail after its execution finishes.

Error message is “first != second” by default. Additional explanation can be supplied in the message.

Parameters:
  • first – The first object to compare.
  • second – The second object to compare.
  • msg – A string that adds additional info about the failure.
  • extras – An optional field for extra information to be included in test result.
mobly.expects.expect_false(condition, msg, extras=None)[source]

Expects an expression evaluates to False.

If the expectation is not met, the test is marked as fail after its execution finishes.

Parameters:
  • expr – The expression that is evaluated.
  • msg – A string explaining the details in case of failure.
  • extras – An optional field for extra information to be included in test result.
mobly.expects.expect_no_raises(message=None, extras=None)[source]

Expects no exception is raised in a context.

If the expectation is not met, the test is marked as fail after its execution finishes.

A default message is added to the exception details.

Parameters:
  • message – string, custom message to add to exception’s details.
  • extras – An optional field for extra information to be included in test result.
mobly.expects.expect_true(condition, msg, extras=None)[source]

Expects an expression evaluates to True.

If the expectation is not met, the test is marked as fail after its execution finishes.

Parameters:
  • expr – The expression that is evaluated.
  • msg – A string explaining the details in case of failure.
  • extras – An optional field for extra information to be included in test result.

mobly.keys module

class mobly.keys.Config[source]

Bases: enum.Enum

The reserved keywordss used in configurations.

key_log_path = 'LogPath'
key_mobly_params = 'MoblyParams'
key_testbed = 'TestBeds'
key_testbed_controllers = 'Controllers'
key_testbed_name = 'Name'
key_testbed_test_params = 'TestParams'

mobly.logger module

mobly.logger.create_latest_log_alias(actual_path, alias)[source]

Creates a symlink to the latest test run logs.

Parameters:
  • actual_path – string, the source directory where the latest test run’s logs are.
  • alias – string, the name of the directory to contain the latest log files.
mobly.logger.epoch_to_log_line_timestamp(epoch_time, time_zone=None)[source]

Converts an epoch timestamp in ms to log line timestamp format, which is readible for humans.

Parameters:
  • epoch_time – integer, an epoch timestamp in ms.
  • time_zone – instance of tzinfo, time zone information. Using pytz rather than python 3.2 time_zone implementation for python 2 compatibility reasons.
Returns:

A string that is the corresponding timestamp in log line timestamp format.

mobly.logger.get_log_file_timestamp(delta=None)[source]

Returns a timestamp in the format used for log file names.

Default is current time. If a delta is set, the return value will be the current time offset by delta seconds.

Parameters:delta – Number of seconds to offset from current time; can be negative.
Returns:A timestamp in log filen name format with an offset.
mobly.logger.get_log_line_timestamp(delta=None)[source]

Returns a timestamp in the format used by log lines.

Default is current time. If a delta is set, the return value will be the current time offset by delta seconds.

Parameters:delta – Number of seconds to offset from current time; can be negative.
Returns:A timestamp in log line format with an offset.
mobly.logger.is_valid_logline_timestamp(timestamp)[source]
mobly.logger.kill_test_logger(logger)[source]

Cleans up a test logger object by removing all of its handlers.

Parameters:logger – The logging object to clean up.
mobly.logger.logline_timestamp_comparator(t1, t2)[source]

Comparator for timestamps in logline format.

Parameters:
  • t1 – Timestamp in logline format.
  • t2 – Timestamp in logline format.
Returns:

-1 if t1 < t2; 1 if t1 > t2; 0 if t1 == t2.

mobly.logger.normalize_log_line_timestamp(log_line_timestamp)[source]

Replace special characters in log line timestamp with normal characters.

Deprecated since version 1.10: This method is obsolete with the more general sanitize_filename method and is only kept for backwards compatibility. In a future update, this method may be removed.

Parameters:log_line_timestamp – A string in the log line timestamp format. Obtained with get_log_line_timestamp.
Returns:A string representing the same time as input timestamp, but without special characters.
mobly.logger.sanitize_filename(filename)[source]

Sanitizes a filename for various operating systems.

Parameters:filename – string, the filename to sanitize.
Returns:A string that is safe to use as a filename on various operating systems.
mobly.logger.setup_test_logger(log_path, prefix=None, alias='latest')[source]

Customizes the root logger for a test run.

In addition to configuring the Mobly logging handlers, this also sets two attributes on the logging module for the output directories:

root_output_path: path to the directory for the entire test run. log_path: same as root_output_path outside of a test class run. In the

context of a test class run, this is the output directory for files specific to a test class.
Parameters:
  • log_path – string, the location of the report file.
  • prefix – optional string, a prefix for each log line in terminal.
  • alias – optional string, The name of the alias to use for the latest log directory. If a falsy value is provided, then the alias directory will not be created, which is useful to save storage space when the storage system (e.g. ZIP files) does not properly support shortcut/symlinks.

mobly.records module

This module has classes for test result collection, and test result output.

class mobly.records.ControllerInfoRecord(test_class, controller_name, info)[source]

Bases: object

A record representing the controller info in test results.

KEY_CONTROLLER_INFO = 'Controller Info'
KEY_CONTROLLER_NAME = 'Controller Name'
KEY_TEST_CLASS = 'Test Class'
KEY_TIMESTAMP = 'Timestamp'
to_dict()[source]
exception mobly.records.Error[source]

Bases: Exception

Raised for errors in records.

class mobly.records.ExceptionRecord(e, position=None)[source]

Bases: object

A record representing exception objects in TestResultRecord.

exception

Exception object, the original Exception.

stacktrace

string, stacktrace of the Exception.

extras

optional serializable, this corresponds to the TestSignal.extras field.

position

string, an optional label specifying the position where the Exception ocurred.

to_dict()[source]
class mobly.records.TestResult[source]

Bases: object

A class that contains metrics of a test run.

This class is essentially a container of TestResultRecord objects.

requested

A list of strings, each is the name of a test requested by user.

failed

A list of records for tests failed.

executed

A list of records for tests that were actually executed.

passed

A list of records for tests passed.

skipped

A list of records for tests skipped.

error

A list of records for tests with error result token.

controller_info

list of ControllerInfoRecord.

add_class_error(test_record)[source]

Add a record to indicate a test class has failed before any test could execute.

This is only called before any test is actually executed. So it only adds an error entry that describes why the class failed to the tally and does not affect the total number of tests requrested or exedcuted.

Parameters:test_record – A TestResultRecord object for the test class.
add_controller_info_record(controller_info_record)[source]

Adds a controller info record to results.

This can be called multiple times for each test class.

Parameters:controller_info_record – ControllerInfoRecord object to be added to the result.
add_record(record)[source]

Adds a test record to test result.

A record is considered executed once it’s added to the test result.

Adding the record finalizes the content of a record, so no change should be made to the record afterwards.

Parameters:record – A test record object to add.
is_all_pass

True if no tests failed or threw errors, False otherwise.

is_test_executed(test_name)[source]

Checks if a specific test has been executed.

Parameters:test_name – string, the name of the test to check.
Returns:True if the test has been executed according to the test result, False otherwise.
requested_test_names_dict()[source]

Gets the requested test names of a test run in a dict format.

Note a test can be requested multiple times, so there can be duplicated values

Returns:A dict with a key and the list of strings.
summary_dict()[source]

Gets a dictionary that summarizes the stats of this test result.

The summary provides the counts of how many tests fall into each category, like ‘Passed’, ‘Failed’ etc.

Returns:A dictionary with the stats of this test result.
summary_str()[source]

Gets a string that summarizes the stats of this test result.

The summary provides the counts of how many tests fall into each category, like ‘Passed’, ‘Failed’ etc.

Format of the string is:
Requested <int>, Executed <int>, …
Returns:A summary string of this test result.
class mobly.records.TestResultEnums[source]

Bases: object

Enums used for TestResultRecord class.

Includes the tokens to mark test result with, and the string names for each field in TestResultRecord.

RECORD_BEGIN_TIME = 'Begin Time'
RECORD_CLASS = 'Test Class'
RECORD_DETAILS = 'Details'
RECORD_END_TIME = 'End Time'
RECORD_EXTRAS = 'Extras'
RECORD_EXTRA_ERRORS = 'Extra Errors'
RECORD_NAME = 'Test Name'
RECORD_POSITION = 'Position'
RECORD_RESULT = 'Result'
RECORD_STACKTRACE = 'Stacktrace'
RECORD_UID = 'UID'
TEST_RESULT_ERROR = 'ERROR'
TEST_RESULT_FAIL = 'FAIL'
TEST_RESULT_PASS = 'PASS'
TEST_RESULT_SKIP = 'SKIP'
class mobly.records.TestResultRecord(t_name, t_class=None)[source]

Bases: object

A record that holds the information of a single test.

The record object holds all information of a test, including all the exceptions occurred during the test.

A test can terminate for two reasons:
  1. the test function executes to the end and completes naturally.
  2. the test is terminated by an exception, which we call “termination signal”.

The termination signal is treated differently. Its content are extracted into first-tier attributes of the record object, like details and stacktrace, for easy consumption.

Note the termination signal is not always an error, it can also be explicit pass signal or abort/skip signals.

test_name

string, the name of the test.

begin_time

Epoch timestamp of when the test started.

end_time

Epoch timestamp of when the test ended.

uid

Unique identifier of a test.

termination_signal

ExceptionRecord, the main exception of the test.

extra_errors

OrderedDict, all exceptions occurred during the entire test lifecycle. The order of occurrence is preserved.

result

TestResultEnum.TEAT_RESULT_*, PASS/FAIL/SKIP.

add_error(position, e)[source]

Add extra error happened during a test.

If the test has passed or skipped, this will mark the test result as ERROR.

If an error is added the test record, the record’s result is equivalent to the case where an uncaught exception happened.

If the test record has not recorded any error, the newly added error would be the main error of the test record. Otherwise the newly added error is added to the record’s extra errors.

Parameters:
  • position – string, where this error occurred, e.g. ‘teardown_test’.
  • e – An exception or a signals.ExceptionRecord object.
details

String description of the cause of the test’s termination.

Note a passed test can have this as well due to the explicit pass signal. If the test passed implicitly, this field would be None.

extras

User defined extra information of the test result.

Must be serializable.

stacktrace

The stacktrace string for the exception that terminated the test.

test_begin()[source]

Call this when the test begins execution.

Sets the begin_time of this record.

test_error(e=None)[source]

To mark the test as error in this record.

Parameters:e – An exception object.
test_fail(e=None)[source]

To mark the test as failed in this record.

Only test_fail does instance check because we want ‘assert xxx’ to also fail the test same way assert_true does.

Parameters:e – An exception object. It can be an instance of AssertionError or mobly.base_test.TestFailure.
test_pass(e=None)[source]

To mark the test as passed in this record.

Parameters:e – An instance of mobly.signals.TestPass.
test_skip(e=None)[source]

To mark the test as skipped in this record.

Parameters:e – An instance of mobly.signals.TestSkip.
to_dict()[source]

Gets a dictionary representating the content of this class.

Returns:A dictionary representating the content of this class.
update_record()[source]

Updates the content of a record.

Several display fields like “details” and “stacktrace” need to be updated based on the content of the record object.

As the content of the record change, call this method to update all the appropirate fields.

class mobly.records.TestSummaryEntryType[source]

Bases: enum.Enum

Constants used to identify the type of entries in test summary file.

Test summary file contains multiple yaml documents. In order to parse this file efficiently, the write adds the type of each entry when it writes the entry to the file.

The idea is similar to how TestResult.json_str categorizes different sections of a TestResult object in the serialized format.

CONTROLLER_INFO = 'ControllerInfo'
RECORD = 'Record'
SUMMARY = 'Summary'
TEST_NAME_LIST = 'TestNameList'
USER_DATA = 'UserData'
class mobly.records.TestSummaryWriter(path)[source]

Bases: object

Writer for the test result summary file of a test run.

For each test run, a writer is created to stream test results to the summary file on disk.

The serialization and writing of the TestResult object is intentionally kept out of TestResult class and put in this class. Because TestResult can be operated on by suites, like + operation, and it is difficult to guarantee the consistency between TestResult in memory and the files on disk. Also, this separation makes it easier to provide a more generic way for users to consume the test summary, like via a database instead of a file.

dump(content, entry_type)[source]

Dumps a dictionary as a yaml document to the summary file.

Each call to this method dumps a separate yaml document to the same summary file associated with a test run.

The content of the dumped dictionary has an extra field TYPE that specifies the type of each yaml document, which is the flag for parsers to identify each document.

Parameters:
  • content – dictionary, the content to serialize and write.
  • entry_type – a member of enum TestSummaryEntryType.
Raises:

recoreds.Error – An invalid entry type is passed in.

mobly.records.uid(uid)[source]

Decorator specifying the unique identifier (UID) of a test case.

The UID will be recorded in the test’s record when executed by Mobly.

If you use any other decorator for the test method, you may want to use this as the outer-most one.

Note a common UID system is the Universal Unitque Identifier (UUID), but we are not limiting people to use UUID, hence the more generic name UID.

Parameters:uid – string, the uid for the decorated test function.

mobly.runtime_test_info module

class mobly.runtime_test_info.RuntimeTestInfo(test_name, log_path, record)[source]

Bases: object

Container class for runtime information of a test or test stage.

One object corresponds to one test. This is meant to be a read-only class.

This also applies to test stages like setup_class, which has its own runtime info but is not part of any single test.

name

string, name of the test.

signature

string, an identifier of the test, a combination of test name and begin time.

record

TestResultRecord, the current test result record. This changes as the test’s execution progresses.

output_path

string, path to the test’s output directory. It’s created upon accessing.

name
output_path
record
signature

mobly.signals module

This module is where all the test signal classes and related utilities live.

exception mobly.signals.ControllerError[source]

Bases: Exception

Raised when an error occurred in controller classes.

exception mobly.signals.TestAbortAll(details, extras=None)[source]

Bases: mobly.signals.TestAbortSignal

Raised when all subsequent tests should be aborted.

exception mobly.signals.TestAbortClass(details, extras=None)[source]

Bases: mobly.signals.TestAbortSignal

Raised when all subsequent tests within the same test class should be aborted.

exception mobly.signals.TestAbortSignal(details, extras=None)[source]

Bases: mobly.signals.TestSignal

Base class for abort signals.

exception mobly.signals.TestError(details, extras=None)[source]

Bases: mobly.signals.TestSignal

Raised when a test has an unexpected error.

exception mobly.signals.TestFailure(details, extras=None)[source]

Bases: mobly.signals.TestSignal

Raised when a test has failed.

exception mobly.signals.TestPass(details, extras=None)[source]

Bases: mobly.signals.TestSignal

Raised when a test has passed.

exception mobly.signals.TestSignal(details, extras=None)[source]

Bases: Exception

Base class for all test result control signals. This is used to signal the result of a test.

details

A string that describes the reason for raising this signal.

extras

A json-serializable data type to convey extra information about a test result.

exception mobly.signals.TestSignalError[source]

Bases: Exception

Raised when an error occurs inside a test signal.

exception mobly.signals.TestSkip(details, extras=None)[source]

Bases: mobly.signals.TestSignal

Raised when a test has been skipped.

mobly.suite_runner module

Runner for Mobly test suites.

To create a test suite, call suite_runner.run_suite() with one or more individual test classes. For example:

from mobly import suite_runner

from my.test.lib import foo_test
from my.test.lib import bar_test
...
if __name__ == '__main__':
    suite_runner.run_suite(foo_test.FooTest, bar_test.BarTest)
exception mobly.suite_runner.Error[source]

Bases: Exception

mobly.suite_runner.compute_selected_tests(test_classes, selected_tests)[source]

Computes tests to run for each class from selector strings.

This function transforms a list of selector strings (such as FooTest or FooTest.test_method_a) to a dict where keys are test_name classes, and values are lists of selected tests in those classes. None means all tests in that class are selected.

Parameters:
  • test_classes – list of strings, names of all the classes that are part of a suite.
  • selected_tests

    list of strings, list of tests to execute. If empty, all classes test_classes are selected. E.g.

    [
        'FooTest',
        'BarTest',
        'BazTest.test_method_a',
        'BazTest.test_method_b'
    ]
    
Returns:

Identifiers for TestRunner. Keys are test class names; valures

are lists of test names within class. E.g. the example in selected_tests would translate to:

{
    FooTest: None,
    BarTest: None,
    BazTest: ['test_method_a', 'test_method_b']
}

This dict is easy to consume for TestRunner.

Return type:

dict

mobly.suite_runner.run_suite(test_classes, argv=None)[source]

Executes multiple test classes as a suite.

This is the default entry point for running a test suite script file directly.

Parameters:
  • test_classes – List of python classes containing Mobly tests.
  • argv – A list that is then parsed as cli args. If None, defaults to cli input.

mobly.test_runner module

exception mobly.test_runner.Error[source]

Bases: Exception

class mobly.test_runner.TestRunner(log_dir, testbed_name)[source]

Bases: object

The class that instantiates test classes, executes tests, and report results.

One TestRunner instance is associated with one specific output folder and testbed. TestRunner.run() will generate a single set of output files and results for all tests that have been added to this runner.

self.results

The test result object used to record the results of this test run.

add_test_class(config, test_class, tests=None, name_suffix=None)[source]

Adds tests to the execution plan of this TestRunner.

Parameters:
  • config – config_parser.TestRunConfig, configuration to execute this test class with.
  • test_class – class, test class to execute.
  • tests – list of strings, optional list of test names within the class to execute.
  • name_suffix – string, suffix to append to the class name for reporting. This is used for differentiating the same class executed with different parameters in a suite.
Raises:

Error – if the provided config has a log_path or testbed_name which differs from the arguments provided to this TestRunner’s constructor.

mobly_logger(alias='latest')[source]

Starts and stops a logging context for a Mobly test run.

Parameters:alias – optional string, the name of the latest log alias directory to create. If a falsy value is specified, then the directory will not be created.
Yields:The host file path where the logs for the test run are stored.
run()[source]

Executes tests.

This will instantiate controller and test classes, execute tests, and print a summary.

Raises:Error – if no tests have previously been added to this runner using add_test_class(…).
mobly.test_runner.main(argv=None)[source]

Execute the test class in a test module.

This is the default entry point for running a test script file directly. In this case, only one test class in a test script is allowed.

To make your test script executable, add the following to your file:

from mobly import test_runner
...
if __name__ == '__main__':
    test_runner.main()

If you want to implement your own cli entry point, you could use function execute_one_test_class(test_class, test_config, test_identifier)

Parameters:argv – A list that is then parsed as cli args. If None, defaults to cli input.
mobly.test_runner.parse_mobly_cli_args(argv)[source]

Parses cli args that are consumed by Mobly.

This is the arg parsing logic for the default test_runner.main entry point.

Multiple arg parsers can be applied to the same set of cli input. So you can use this logic in addition to any other args you want to parse. This function ignores the args that don’t apply to default test_runner.main.

Parameters:argv – A list that is then parsed as cli args. If None, defaults to cli input.
Returns:Namespace containing the parsed args.

mobly.utils module

exception mobly.utils.Error[source]

Bases: Exception

Raised when an error occurs in a util

mobly.utils.abs_path(path)[source]

Resolve the ‘.’ and ‘~’ in a path to get the absolute path.

Parameters:path – The path to expand.
Returns:The absolute path of the input path.
mobly.utils.cli_cmd_to_string(args)[source]

Converts a cmd arg list to string.

Parameters:args – list of strings, the arguments of a command.
Returns:String representation of the command.
mobly.utils.concurrent_exec(func, param_list, max_workers=30, raise_on_exception=False)[source]

Executes a function with different parameters pseudo-concurrently.

This is basically a map function. Each element (should be an iterable) in the param_list is unpacked and passed into the function. Due to Python’s GIL, there’s no true concurrency. This is suited for IO-bound tasks.

Parameters:
  • func – The function that parforms a task.
  • param_list – A list of iterables, each being a set of params to be passed into the function.
  • max_workers – int, the number of workers to use for parallelizing the tasks. By default, this is 30 workers.
  • raise_on_exception – bool, raises all of the task failures if any of the tasks failed if True. By default, this is False.
Returns:

A list of return values from each function execution. If an execution caused an exception, the exception object will be the corresponding result.

Raises:

RuntimeError – If executing any of the tasks failed and raise_on_exception is True.

mobly.utils.create_alias(target_path, alias_path)[source]

Creates an alias at ‘alias_path’ pointing to the file ‘target_path’.

On Unix, this is implemented via symlink. On Windows, this is done by creating a Windows shortcut file.

Parameters:
  • target_path – Destination path that the alias should point to.
  • alias_path – Path at which to create the new alias.
mobly.utils.create_dir(path)[source]

Creates a directory if it does not exist already.

Parameters:path – The path of the directory to create.
mobly.utils.epoch_to_human_time(epoch_time)[source]

Converts an epoch timestamp to human readable time.

This essentially converts an output of get_current_epoch_time to an output of get_current_human_time

Parameters:epoch_time – An integer representing an epoch timestamp in milliseconds.
Returns:A time string representing the input time. None if input param is invalid.
mobly.utils.find_field(item_list, cond, comparator, target_field)[source]

Finds the value of a field in a dict object that satisfies certain conditions.

Parameters:
  • item_list – A list of dict objects.
  • cond – A param that defines the condition.
  • comparator – A function that checks if an dict satisfies the condition.
  • target_field – Name of the field whose value to be returned if an item satisfies the condition.
Returns:

Target value or None if no item satisfies the condition.

mobly.utils.find_files(paths, file_predicate)[source]

Locate files whose names and extensions match the given predicate in the specified directories.

Parameters:
  • paths – A list of directory paths where to find the files.
  • file_predicate – A function that returns True if the file name and extension are desired.
Returns:

A list of files that match the predicate.

mobly.utils.find_subclass_in_module(base_class, module)[source]

Finds the single subclass of the given base class in the given module.

Parameters:
  • base_class – class, the base class to look for a subclass of in the module.
  • module – module, the module to look for the single subclass in.
Returns:

The single subclass of the given base class.

Raises:

ValueError – If the number of subclasses found was not exactly one.

mobly.utils.find_subclasses_in_module(base_classes, module)[source]

Finds the subclasses of the given classes in the given module.

Parameters:
  • base_classes – list of classes, the base classes to look for the subclasses of in the module.
  • module – module, the module to look for the subclasses in.
Returns:

A list of all of the subclasses found in the module.

mobly.utils.get_available_host_port()[source]

Gets a host port number available for adb forward.

Returns:An integer representing a port number on the host available for adb forward.
Raises:Error – when no port is found after MAX_PORT_ALLOCATION_RETRY times.
mobly.utils.get_current_epoch_time()[source]

Current epoch time in milliseconds.

Returns:An integer representing the current epoch time in milliseconds.
mobly.utils.get_current_human_time()[source]

Returns the current time in human readable format.

Returns:Min:Sec format.
Return type:The current time stamp in Month-Day-Year Hour
mobly.utils.get_settable_properties(cls)[source]

Gets the settable properties of a class.

Only returns the explicitly defined properties with setters.

Parameters:cls – A class in Python.
mobly.utils.get_timezone_olson_id()[source]

Return the Olson ID of the local (non-DST) timezone.

Returns:A string representing one of the Olson IDs of the local (non-DST) timezone.
mobly.utils.grep(regex, output)[source]

Similar to linux’s grep, this returns the line in an output stream that matches a given regex pattern.

It does not rely on the grep binary and is not sensitive to line endings, so it can be used cross-platform.

Parameters:
  • regex – string, a regex that matches the expected pattern.
  • output – byte string, the raw output of the adb cmd.
Returns:

A list of strings, all of which are output lines that matches the regex pattern.

mobly.utils.load_file_to_base64_str(f_path)[source]

Loads the content of a file into a base64 string.

Parameters:f_path – full path to the file including the file name.
Returns:A base64 string representing the content of the file in utf-8 encoding.
mobly.utils.rand_ascii_str(length)[source]

Generates a random string of specified length, composed of ascii letters and digits.

Parameters:length – The number of characters in the string.
Returns:The random string generated.
mobly.utils.run_command(cmd, stdout=None, stderr=None, shell=False, timeout=None, cwd=None, env=None)[source]

Runs a command in a subprocess.

This function is very similar to subprocess.check_output. The main difference is that it returns the return code and std error output as well as supporting a timeout parameter.

Parameters:
  • cmd – string or list of strings, the command to run. See subprocess.Popen() documentation.
  • stdout – file handle, the file handle to write std out to. If None is given, then subprocess.PIPE is used. See subprocess.Popen() documentation.
  • stdee – file handle, the file handle to write std err to. If None is given, then subprocess.PIPE is used. See subprocess.Popen() documentation.
  • shell – bool, True to run this command through the system shell, False to invoke it directly. See subprocess.Popen() docs.
  • timeout – float, the number of seconds to wait before timing out. If not specified, no timeout takes effect.
  • cwd – string, the path to change the child’s current directory to before it is executed. Note that this directory is not considered when searching the executable, so you can’t specify the program’s path relative to cwd.
  • env – dict, a mapping that defines the environment variables for the new process. Default behavior is inheriting the current process’ environment.
Returns:

A 3-tuple of the consisting of the return code, the std output, and the

std error.

Raises:

psutil.TimeoutExpired – The command timed out.

mobly.utils.start_standing_subprocess(cmd, shell=False, env=None)[source]

Starts a long-running subprocess.

This is not a blocking call and the subprocess started by it should be explicitly terminated with stop_standing_subprocess.

For short-running commands, you should use subprocess.check_call, which blocks.

Parameters:
  • cmd – string, the command to start the subprocess with.
  • shell – bool, True to run this command through the system shell, False to invoke it directly. See subprocess.Proc() docs.
  • env – dict, a custom environment to run the standing subprocess. If not specified, inherits the current environment. See subprocess.Popen() docs.
Returns:

The subprocess that was started.

mobly.utils.stop_standing_subprocess(proc)[source]

Stops a subprocess started by start_standing_subprocess.

Before killing the process, we check if the process is running, if it has terminated, Error is raised.

Catches and ignores the PermissionError which only happens on Macs.

Parameters:proc – Subprocess to terminate.
Raises:Error – if the subprocess could not be stopped.
mobly.utils.wait_for_standing_subprocess(proc, timeout=None)[source]

Waits for a subprocess started by start_standing_subprocess to finish or times out.

Propagates the exception raised by the subprocess.wait(.) function. The subprocess.TimeoutExpired exception is raised if the process timed-out rather than terminating.

If no exception is raised: the subprocess terminated on its own. No need to call stop_standing_subprocess() to kill it.

If an exception is raised: the subprocess is still alive - it did not terminate. Either call stop_standing_subprocess() to kill it, or call wait_for_standing_subprocess() to keep waiting for it to terminate on its own.

If the corresponding subprocess command generates a large amount of output and this method is called with a timeout value, then the command can hang indefinitely. See http://go/pylib/subprocess.html#subprocess.Popen.wait

This function does not support Python 2.

Parameters:
  • p – Subprocess to wait for.
  • timeout – An integer number of seconds to wait before timing out.

Module contents