sascha/devlib
1
0
Fork 0
mirror of https://github.com/ARM-software/devlib.git synced 2025-09-04 11:01:53 +01:00
Code Issues Releases Wiki Activity

devlib initial commit.

Browse Source
This commit is contained in:
Sergei Trofimov
2015-10-09 09:30:04 +01:00
commit 4e6afe960b
64 changed files with 8938 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
devlib/__init__.py Normal file
View File

@@ -0,0 +1,18 @@
from devlib.target import Target, LinuxTarget, AndroidTarget, LocalLinuxTarget
from devlib.host import PACKAGE_BIN_DIRECTORY
from devlib.exception import DevlibError, TargetError, HostError, TargetNotRespondingError
from devlib.module import Module, HardRestModule, BootModule, FlashModule
from devlib.module import get_module, register_module
from devlib.platform import Platform
from devlib.platform.arm import TC2, Juno, JunoEnergyInstrument
from devlib.instrument import Instrument, InstrumentChannel, Measurement, MeasurementsCsv
from devlib.instrument import MEASUREMENT_TYPES, INSTANTANEOUS, CONTINUOUS
from devlib.instrument.daq import DaqInstrument
from devlib.instrument.energy_probe import EnergyProbeInstrument
from devlib.instrument.hwmon import HwmonInstrument
from devlib.instrument.netstats import NetstatsInstrument
from devlib.trace.ftrace import FtraceCollector

348
devlib/bin/LICENSE.busybox Normal file
View File

@@ -0,0 +1,348 @@
--- A note on GPL versions
BusyBox is distributed under version 2 of the General Public License (included
in its entirety, below). Version 2 is the only version of this license which
this version of BusyBox (or modified versions derived from this one) may be
distributed under.
------------------------------------------------------------------------
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.
51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Library General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Library General
Public License instead of this License.

39
devlib/bin/LICENSE.trace-cmd Normal file
View File

@@ -0,0 +1,39 @@
Included trace-cmd binaries are Free Software ditributed under GPLv2:
/*
* Copyright (C) 2009, 2010 Red Hat Inc, Steven Rostedt <srostedt@redhat.com>
*
* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; version 2 of the License (not later!)
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, see <http://www.gnu.org/licenses>
*
* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*/
The full text of the license may be viewed here:
http://www.gnu.org/licenses/gpl-2.0.html
Source code for trace-cmd may be obtained here:
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git
Binaries included here contain modifications by ARM that, at the time of writing,
have not yet made it into the above repository. The patches for these modifications
are available here:
http://article.gmane.org/gmane.linux.kernel/1869111
http://article.gmane.org/gmane.linux.kernel/1869112

BIN
devlib/bin/arm64/busybox Executable file
View File

Binary file not shown.

BIN
devlib/bin/arm64/readenergy Executable file
View File

Binary file not shown.

BIN
devlib/bin/arm64/trace-cmd Executable file
View File

Binary file not shown.

BIN
devlib/bin/armeabi/busybox Executable file
View File

Binary file not shown.

BIN
devlib/bin/armeabi/trace-cmd Executable file
View File

Binary file not shown.

40
devlib/exception.py Normal file
View File

@@ -0,0 +1,40 @@
# Copyright 2013-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from devlib.utils.misc import TimeoutError # NOQA pylint: disable=W0611
class DevlibError(Exception):
"""Base class for all Workload Automation exceptions."""
pass
class TargetError(DevlibError):
"""An error has occured on the target"""
pass
class TargetNotRespondingError(DevlibError):
"""The target is unresponsive."""
def __init__(self, target):
super(TargetNotRespondingError, self).__init__('Target {} is not responding.'.format(target))
class HostError(DevlibError):
"""An error has occured on the host"""
pass

80
devlib/host.py Normal file
View File

@@ -0,0 +1,80 @@
# Copyright 2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import os
import shutil
import subprocess
import logging
from getpass import getpass
from devlib.exception import TargetError
from devlib.utils.misc import check_output
PACKAGE_BIN_DIRECTORY = os.path.join(os.path.dirname(__file__), 'bin')
class LocalConnection(object):
name = 'local'
def __init__(self, timeout=10, keep_password=True, unrooted=False):
self.logger = logging.getLogger('local_connection')
self.timeout = timeout
self.keep_password = keep_password
self.unrooted = unrooted
self.password = None
def push(self, source, dest, timeout=None, as_root=False): # pylint: disable=unused-argument
self.logger.debug('cp {} {}'.format(source, dest))
shutil.copy(source, dest)
def pull(self, source, dest, timeout=None, as_root=False): # pylint: disable=unused-argument
self.logger.debug('cp {} {}'.format(source, dest))
shutil.copy(source, dest)
def execute(self, command, timeout=None, check_exit_code=True, as_root=False):
self.logger.debug(command)
if as_root:
if self.unrooted:
raise TargetError('unrooted')
password = self._get_password()
command = 'echo \'{}\' | sudo -S '.format(password) + command
ignore = None if check_exit_code else 'all'
try:
return check_output(command, shell=True, timeout=timeout, ignore=ignore)[0]
except subprocess.CalledProcessError as e:
raise TargetError(e)
def background(self, command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, as_root=False):
if as_root:
if self.unrooted:
raise TargetError('unrooted')
password = self._get_password()
command = 'echo \'{}\' | sudo -S '.format(password) + command
return subprocess.Popen(command, stdout=stdout, stderr=stderr, shell=True)
def close(self):
pass
def cancel_running_command(self):
pass
def _get_password(self):
if self.password:
return self.password
password = getpass('sudo password:')
if self.keep_password:
self.password = password
return password

225
devlib/instrument/__init__.py Normal file
View File

@@ -0,0 +1,225 @@
# Copyright 2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import csv
import logging
from devlib.utils.types import numeric
# Channel modes describe what sort of measurement the instrument supports.
# Values must be powers of 2
INSTANTANEOUS = 1
CONTINUOUS = 2
class MeasurementType(tuple):
__slots__ = []
def __new__(cls, name, units, category=None):
return tuple.__new__(cls, (name, units, category))
@property
def name(self):
return tuple.__getitem__(self, 0)
@property
def units(self):
return tuple.__getitem__(self, 1)
@property
def category(self):
return tuple.__getitem__(self, 2)
def __getitem__(self, item):
raise TypeError()
def __cmp__(self, other):
if isinstance(other, MeasurementType):
other = other.name
return cmp(self.name, other)
def __str__(self):
return self.name
__repr__ = __str__
# Standard measures
_measurement_types = [
MeasurementType('time', 'seconds'),
MeasurementType('temperature', 'degrees'),
MeasurementType('power', 'watts', 'power/energy'),
MeasurementType('voltage', 'volts', 'power/energy'),
MeasurementType('current', 'amps', 'power/energy'),
MeasurementType('energy', 'joules', 'power/energy'),
MeasurementType('tx', 'bytes', 'data transfer'),
MeasurementType('rx', 'bytes', 'data transfer'),
MeasurementType('tx/rx', 'bytes', 'data transfer'),
]
MEASUREMENT_TYPES = {m.name: m for m in _measurement_types}
class Measurement(object):
__slots__ = ['value', 'channel']
@property
def name(self):
return '{}_{}'.format(self.channel.site, self.channel.kind)
@property
def units(self):
return self.channel.units
def __init__(self, value, channel):
self.value = value
self.channel = channel
def __cmp__(self, other):
if isinstance(other, Measurement):
return cmp(self.value, other.value)
else:
return cmp(self.value, other)
def __str__(self):
if self.units:
return '{}: {} {}'.format(self.name, self.value, self.units)
else:
return '{}: {}'.format(self.name, self.value)
__repr__ = __str__
class MeasurementsCsv(object):
def __init__(self, path, channels):
self.path = path
self.channels = channels
self._fh = open(path, 'rb')
def measurements(self):
return list(self.itermeasurements())
def itermeasurements(self):
self._fh.seek(0)
reader = csv.reader(self._fh)
reader.next() # headings
for row in reader:
values = map(numeric, row)
yield [Measurement(v, c) for (v, c) in zip(values, self.channels)]
class InstrumentChannel(object):
@property
def label(self):
return '{}_{}'.format(self.site, self.kind)
@property
def kind(self):
return self.measurement_type.name
@property
def units(self):
return self.measurement_type.units
def __init__(self, name, site, measurement_type, **attrs):
self.name = name
self.site = site
if isinstance(measurement_type, MeasurementType):
self.measurement_type = measurement_type
else:
try:
self.measurement_type = MEASUREMENT_TYPES[measurement_type]
except KeyError:
raise ValueError('Unknown measurement type: {}'.format(measurement_type))
for atname, atvalue in attrs.iteritems():
setattr(self, atname, atvalue)
def __str__(self):
if self.name == self.label:
return 'CHAN({})'.format(self.label)
else:
return 'CHAN({}, {})'.format(self.name, self.label)
__repr__ = __str__
class Instrument(object):
mode = 0
def __init__(self, target):
self.target = target
self.logger = logging.getLogger(self.__class__.__name__)
self.channels = {}
self.active_channels = []
# channel management
def list_channels(self):
return self.channels.values()
def get_channels(self, measure):
if hasattr(measure, 'name'):
measure = measure.name
return [c for c in self.channels if c.measure.name == measure]
def add_channel(self, site, measure, name=None, **attrs):
if name is None:
name = '{}_{}'.format(site, measure)
chan = InstrumentChannel(name, site, measure, **attrs)
self.channels[chan.label] = chan
# initialization and teardown
def setup(self, *args, **kwargs):
pass
def teardown(self):
pass
def reset(self, sites=None, kinds=None):
if kinds is None and sites is None:
self.active_channels = sorted(self.channels.values(), key=lambda x: x.label)
else:
if isinstance(sites, basestring):
sites = [sites]
if isinstance(kinds, basestring):
kinds = [kinds]
self.active_channels = []
for chan in self.channels.values():
if (kinds is None or chan.kind in kinds) and \
(sites is None or chan.site in sites):
self.active_channels.append(chan)
# instantaneous
def take_measurement(self):
pass
# continuous
def start(self):
pass
def stop(self):
pass
def get_data(self, outfile):
pass

138
devlib/instrument/daq.py Normal file
View File

@@ -0,0 +1,138 @@
import os
import csv
import tempfile
from itertools import chain
from devlib.instrument import Instrument, MeasurementsCsv, CONTINUOUS
from devlib.exception import HostError
from devlib.utils.misc import unique
try:
from daqpower.client import execute_command, Status
from daqpower.config import DeviceConfiguration, ServerConfiguration
except ImportError, e:
execute_command, Status = None, None
DeviceConfiguration, ServerConfiguration, ConfigurationError = None, None, None
import_error_mesg = e.message
class DaqInstrument(Instrument):
mode = CONTINUOUS
def __init__(self, target, resistor_values, # pylint: disable=R0914
labels=None,
host='localhost',
port=45677,
device_id='Dev1',
v_range=2.5,
dv_range=0.2,
sampling_rate=10000,
channel_map=(0, 1, 2, 3, 4, 5, 6, 7, 16, 17, 18, 19, 20, 21, 22, 23),
):
# pylint: disable=no-member
super(DaqInstrument, self).__init__(target)
self._need_reset = True
if execute_command is None:
raise HostError('Could not import "daqpower": {}'.format(import_error_mesg))
if labels is None:
labels = ['PORT_{}'.format(i) for i in xrange(len(resistor_values))]
if len(labels) != len(resistor_values):
raise ValueError('"labels" and "resistor_values" must be of the same length')
self.server_config = ServerConfiguration(host=host,
port=port)
result = self.execute('list_devices')
if result.status == Status.OK:
if device_id not in result.data:
raise ValueError('Device "{}" is not found on the DAQ server.'.format(device_id))
elif result.status != Status.OKISH:
raise HostError('Problem querying DAQ server: {}'.format(result.message))
self.device_config = DeviceConfiguration(device_id=device_id,
v_range=v_range,
dv_range=dv_range,
sampling_rate=sampling_rate,
resistor_values=resistor_values,
channel_map=channel_map,
labels=labels)
for label in labels:
for kind in ['power', 'voltage']:
self.add_channel(label, kind)
def reset(self, sites=None, kinds=None):
super(DaqInstrument, self).reset(sites, kinds)
self.execute('close')
result = self.execute('configure', config=self.device_config)
if not result.status == Status.OK: # pylint: disable=no-member
raise HostError(result.message)
self._need_reset = False
def start(self):
if self._need_reset:
self.reset()
self.execute('start')
def stop(self):
self.execute('stop')
self._need_reset = True
def get_data(self, outfile): # pylint: disable=R0914
tempdir = tempfile.mkdtemp(prefix='daq-raw-')
self.execute('get_data', output_directory=tempdir)
raw_file_map = {}
for entry in os.listdir(tempdir):
site = os.path.splitext(entry)[0]
path = os.path.join(tempdir, entry)
raw_file_map[site] = path
active_sites = unique([c.site for c in self.active_channels])
file_handles = []
try:
site_readers = {}
for site in active_sites:
try:
site_file = raw_file_map[site]
fh = open(site_file, 'rb')
site_readers[site] = csv.reader(fh)
file_handles.append(fh)
except KeyError:
message = 'Could not get DAQ trace for {}; Obtained traces are in {}'
raise HostError(message.format(site, tempdir))
# The first row is the headers
channel_order = []
for site, reader in site_readers.iteritems():
channel_order.extend(['{}_{}'.format(site, kind)
for kind in reader.next()])
def _read_next_rows():
parts = []
for reader in site_readers.itervalues():
try:
parts.extend(reader.next())
except StopIteration:
parts.extend([None, None])
return list(chain(parts))
with open(outfile, 'wb') as wfh:
field_names = [c.label for c in self.active_channels]
writer = csv.writer(wfh)
writer.writerow(field_names)
raw_row = _read_next_rows()
while any(raw_row):
row = [raw_row[channel_order.index(f)] for f in field_names]
writer.writerow(row)
raw_row = _read_next_rows()
return MeasurementsCsv(outfile, self.active_channels)
finally:
for fh in file_handles:
fh.close()
def teardown(self):
self.execute('close')
def execute(self, command, **kwargs):
return execute_command(self.server_config, command, **kwargs)

116
devlib/instrument/energy_probe.py Normal file
View File

@@ -0,0 +1,116 @@
# Copyright 2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import division
import os
import csv
import signal
import tempfile
import struct
import subprocess
try:
import pandas
except ImportError:
pandas = None
from devlib.instrument import Instrument, CONTINUOUS, MeasurementsCsv
from devlib.exception import HostError
from devlib.utils.misc import which
class EnergyProbeInstrument(Instrument):
mode = CONTINUOUS
def __init__(self, target, resistor_values,
labels=None,
device_entry='/dev/ttyACM0',
):
super(EnergyProbeInstrument, self).__init__(target)
self.resistor_values = resistor_values
if labels is not None:
self.labels = labels
else:
self.labels = ['PORT_{}'.format(i)
for i in xrange(len(resistor_values))]
self.device_entry = device_entry
self.caiman = which('caiman')
if self.caiman is None:
raise HostError('caiman must be installed on the host '
'(see https://github.com/ARM-software/caiman)')
if pandas is None:
self.logger.info("pandas package will significantly speed up this instrument")
self.logger.info("to install it try: pip install pandas")
self.attributes_per_sample = 3
self.bytes_per_sample = self.attributes_per_sample * 4
self.attributes = ['power', 'voltage', 'current']
self.command = None
self.raw_output_directory = None
self.process = None
for label in self.labels:
for kind in self.attributes:
self.add_channel(label, kind)
def reset(self, sites=None, kinds=None):
super(EnergyProbeInstrument, self).reset(sites, kinds)
self.raw_output_directory = tempfile.mkdtemp(prefix='eprobe-caiman-')
parts = ['-r {}:{} '.format(i, int(1000 * rval))
for i, rval in enumerate(self.resistor_values)]
rstring = ''.join(parts)
self.command = '{} -d {} -l {} {}'.format(self.caiman, self.device_entry, rstring, self.raw_output_directory)
def start(self):
self.logger.debug(self.command)
self.process = subprocess.Popen(self.command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
stdin=subprocess.PIPE,
preexec_fn=os.setpgrp,
shell=True)
def stop(self):
os.killpg(self.process.pid, signal.SIGTERM)
def get_data(self, outfile): # pylint: disable=R0914
all_channels = [c.label for c in self.list_channels()]
active_channels = [c.label for c in self.active_channels]
active_indexes = [all_channels.index(ac) for ac in active_channels]
num_of_ports = len(self.resistor_values)
struct_format = '{}I'.format(num_of_ports * self.attributes_per_sample)
not_a_full_row_seen = False
raw_data_file = os.path.join(self.raw_output_directory, '0000000000')
self.logger.debug('Parsing raw data file: {}'.format(raw_data_file))
with open(raw_data_file, 'rb') as bfile:
with open(outfile, 'wb') as wfh:
writer = csv.writer(wfh)
writer.writerow(active_channels)
while True:
data = bfile.read(num_of_ports * self.bytes_per_sample)
if data == '':
break
try:
unpacked_data = struct.unpack(struct_format, data)
row = [unpacked_data[i] / 1000 for i in active_indexes]
writer.writerow(row)
except struct.error:
if not_a_full_row_seen:
self.logger.warn('possibly missaligned caiman raw data, row contained {} bytes'.format(len(data)))
continue
else:
not_a_full_row_seen = True
return MeasurementsCsv(outfile, self.active_channels)

89
devlib/instrument/hwmon.py Normal file
View File

@@ -0,0 +1,89 @@
# Copyright 2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import division
import re
from devlib.instrument import Instrument, Measurement, INSTANTANEOUS
from devlib.exception import TargetError
class HwmonInstrument(Instrument):
name = 'hwmon'
mode = INSTANTANEOUS
# sensor kind --> (meaure, standard unit conversion)
measure_map = {
'temp': ('temperature', lambda x: x / 1000),
'in': ('voltage', lambda x: x / 1000),
'curr': ('current', lambda x: x / 1000),
'power': ('power', lambda x: x / 1000000),
'energy': ('energy', lambda x: x / 1000000),
}
def __init__(self, target):
if not hasattr(target, 'hwmon'):
raise TargetError('Target does not support HWMON')
super(HwmonInstrument, self).__init__(target)
self.logger.debug('Discovering available HWMON sensors...')
for ts in self.target.hwmon.sensors:
try:
ts.get_file('input')
measure = self.measure_map.get(ts.kind)[0]
if measure:
self.logger.debug('\tAdding sensor {}'.format(ts.name))
self.add_channel(_guess_site(ts), measure, name=ts.name, sensor=ts)
else:
self.logger.debug('\tSkipping sensor {} (unknown kind "{}")'.format(ts.name, ts.kind))
except ValueError:
message = 'Skipping sensor {} because it does not have an input file'
self.logger.debug(message.format(ts.name))
continue
def take_measurement(self):
result = []
for chan in self.active_channels:
convert = self.measure_map[chan.sensor.kind][1]
value = convert(chan.sensor.get('input'))
result.append(Measurement(value, chan))
return result
def _guess_site(sensor):
"""
HWMON does not specify a standard for labeling its sensors, or for
device/item split (the implication is that each hwmon device a separate chip
with possibly several sensors on it, but not everyone adheres to that, e.g.,
with some mobile devices splitting a chip's sensors across multiple hwmon
devices. This function processes name/label of the senors to attempt to
identify the best "candidate" for the site to which the sensor belongs.
"""
if sensor.name == sensor.label:
# If no label has been specified for the sensor (in which case, it
# defaults to the sensor's name), assume that the "site" of the sensor
# is identified by the HWMON device
text = sensor.device.name
else:
# If a label has been specified, assume multiple sensors controlled by
# the same device and the label identifies the site.
text = sensor.label
# strip out sensor kind suffix, if any, as that does not indicate a site
for kind in ['volt', 'in', 'curr', 'power', 'energy',
'temp', 'voltage', 'temperature', 'current']:
if kind in text.lower():
regex = re.compile(r'_*{}\d*_*'.format(kind), re.I)
text = regex.sub('', text)
return text.strip()

135
devlib/instrument/netstats/__init__.py Normal file
View File

@@ -0,0 +1,135 @@
import os
import re
import csv
import tempfile
from datetime import datetime
from collections import defaultdict
from itertools import izip_longest
from devlib.instrument import Instrument, MeasurementsCsv, CONTINUOUS
from devlib.exception import TargetError, HostError
from devlib.utils.android import ApkInfo
THIS_DIR = os.path.dirname(__file__)
NETSTAT_REGEX = re.compile(r'I/(?P<tag>netstats-\d+)\(\s*\d*\): (?P<ts>\d+) '
r'"(?P<package>[^"]+)" TX: (?P<tx>\S+) RX: (?P<rx>\S+)')
def extract_netstats(filepath, tag=None):
netstats = []
with open(filepath) as fh:
for line in fh:
match = NETSTAT_REGEX.search(line)
if not match:
continue
if tag and match.group('tag') != tag:
continue
netstats.append((match.group('tag'),
match.group('ts'),
match.group('package'),
match.group('tx'),
match.group('rx')))
return netstats
def netstats_to_measurements(netstats):
measurements = defaultdict(list)
for row in netstats:
tag, ts, package, tx, rx = row # pylint: disable=unused-variable
measurements[package + '_tx'].append(tx)
measurements[package + '_rx'].append(rx)
return measurements
def write_measurements_csv(measurements, filepath):
headers = sorted(measurements.keys())
columns = [measurements[h] for h in headers]
with open(filepath, 'wb') as wfh:
writer = csv.writer(wfh)
writer.writerow(headers)
writer.writerows(izip_longest(*columns))
class NetstatsInstrument(Instrument):
mode = CONTINUOUS
def __init__(self, target, apk=None, service='.TrafficMetricsService'):
"""
Additional paramerter:
:apk: Path to the APK file that contains ``com.arm.devlab.netstats``
package. If not specified, it will be assumed that an APK with
name "netstats.apk" is located in the same directory as the
Python module for the instrument.
:service: Name of the service to be launched. This service must be
present in the APK.
"""
if target.os != 'android':
raise TargetError('netstats insturment only supports Android targets')
if apk is None:
apk = os.path.join(THIS_DIR, 'netstats.apk')
if not os.path.isfile(apk):
raise HostError('APK for netstats instrument does not exist ({})'.format(apk))
super(NetstatsInstrument, self).__init__(target)
self.apk = apk
self.package = ApkInfo(self.apk).package
self.service = service
self.tag = None
self.command = None
self.stop_command = 'am kill {}'.format(self.package)
for package in self.target.list_packages():
self.add_channel(package, 'tx')
self.add_channel(package, 'rx')
def setup(self, force=False, *args, **kwargs):
if self.target.package_is_installed(self.package):
if force:
self.logger.debug('Re-installing {} (forced)'.format(self.package))
self.target.uninstall_package(self.package)
self.target.install(self.apk)
else:
self.logger.debug('{} already present on target'.format(self.package))
else:
self.logger.debug('Deploying {} to target'.format(self.package))
self.target.install(self.apk)
def reset(self, sites=None, kinds=None, period=None): # pylint: disable=arguments-differ
super(NetstatsInstrument, self).reset(sites, kinds)
period_arg, packages_arg = '', ''
self.tag = 'netstats-{}'.format(datetime.now().strftime('%Y%m%d%H%M%s'))
tag_arg = ' --es tag {}'.format(self.tag)
if sites:
packages_arg = ' --es packages {}'.format(','.join(sites))
if period:
period_arg = ' --ei period {}'.format(period)
self.command = 'am startservice{}{}{} {}/{}'.format(tag_arg,
period_arg,
packages_arg,
self.package,
self.service)
self.target.execute(self.stop_command) # ensure the service is not running.
def start(self):
if self.command is None:
raise RuntimeError('reset() must be called before start()')
self.target.execute(self.command)
def stop(self):
self.target.execute(self.stop_command)
def get_data(self, outfile):
raw_log_file = tempfile.mktemp()
self.target.dump_logcat(raw_log_file)
data = extract_netstats(raw_log_file)
measurements = netstats_to_measurements(data)
write_measurements_csv(measurements, outfile)
os.remove(raw_log_file)
return MeasurementsCsv(outfile, self.active_channels)
def teardown(self):
self.target.uninstall_package(self.package)

BIN
devlib/instrument/netstats/netstats.apk Normal file
View File

Binary file not shown.

122
devlib/module/__init__.py Normal file
View File

@@ -0,0 +1,122 @@
# Copyright 2014-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import logging
from inspect import isclass
from devlib.utils.misc import walk_modules
from devlib.utils.types import identifier
__module_cache = {}
class Module(object):
name = None
kind = None
# This is the stage at which the module will be installed. Current valid
# stages are:
# 'early' -- installed when the Target is first created. This should be
# used for modules that do not rely on the main connection
# being established (usually because the commumnitcate with the
# target through some sorto of secondary connection, e.g. via
# serial).
# 'connected' -- installed when a connection to to the target has been
# established. This is the default.
stage = 'connected'
@staticmethod
def probe(target):
raise NotImplementedError()
@classmethod
def install(cls, target, **params):
if cls.kind is not None:
attr_name = identifier(cls.kind)
else:
attr_name = identifier(cls.name)
if hasattr(target, attr_name):
existing_module = getattr(target, attr_name)
existing_name = getattr(existing_module, 'name', str(existing_module))
message = 'Attempting to install module "{}" which already exists (new: {}, existing: {})'
raise ValueError(message.format(attr_name, cls.name, existing_name))
setattr(target, attr_name, cls(target, **params))
def __init__(self, target):
self.target = target
self.logger = logging.getLogger(self.__class__.__name__)
class HardRestModule(Module): # pylint: disable=R0921
kind = 'hard_reset'
def __call__(self):
raise NotImplementedError()
class BootModule(Module): # pylint: disable=R0921
kind = 'boot'
def __call__(self):
raise NotImplementedError()
def update(self, **kwargs):
for name, value in kwargs.iteritems():
if not hasattr(self, name):
raise ValueError('Unknown parameter "{}" for {}'.format(name, self.name))
self.logger.debug('Updating "{}" to "{}"'.format(name, value))
setattr(self, name, value)
class FlashModule(Module):
kind = 'flash'
def __call__(self, image_bundle=None, images=None, boot_config=None):
raise NotImplementedError()
def get_module(mod):
if not __module_cache:
__load_cache()
if isinstance(mod, basestring):
try:
return __module_cache[mod]
except KeyError:
raise ValueError('Module "{}" does not exist'.format(mod))
elif issubclass(mod, Module):
return mod
else:
raise ValueError('Not a valid module: {}'.format(mod))
def register_module(mod):
if not issubclass(mod, Module):
raise ValueError('A module must subclass devlib.Module')
if mod.name is None:
raise ValueError('A module must define a name')
if mod.name in __module_cache:
raise ValueError('Module {} already exists'.format(mod.name))
__module_cache[mod.name] = mod
def __load_cache():
for module in walk_modules('devlib.module'):
for obj in vars(module).itervalues():
if isclass(obj) and issubclass(obj, Module) and obj.name:
register_module(obj)

128
devlib/module/android.py Normal file
View File

@@ -0,0 +1,128 @@
# Copyright 2014-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# pylint: disable=attribute-defined-outside-init
import os
import time
import tarfile
import tempfile
from devlib.module import FlashModule
from devlib.exception import HostError
from devlib.utils.android import fastboot_flash_partition, fastboot_command
from devlib.utils.misc import merge_dicts
class FastbootFlashModule(FlashModule):
name = 'fastboot'
description = """
Enables automated flashing of images using the fastboot utility.
To use this flasher, a set of image files to be flused are required.
In addition a mapping between partitions and image file is required. There are two ways
to specify those requirements:
- Image mapping: In this mode, a mapping between partitions and images is given in the agenda.
- Image Bundle: In This mode a tarball is specified, which must contain all image files as well
as well as a partition file, named ``partitions.txt`` which contains the mapping between
partitions and images.
The format of ``partitions.txt`` defines one mapping per line as such: ::
kernel zImage-dtb
ramdisk ramdisk_image
"""
delay = 0.5
partitions_file_name = 'partitions.txt'
@staticmethod
def probe(target):
return target.os == 'android'
def __call__(self, image_bundle=None, images=None, bootargs=None):
if bootargs:
raise ValueError('{} does not support boot configuration'.format(self.name))
self.prelude_done = False
to_flash = {}
if image_bundle: # pylint: disable=access-member-before-definition
image_bundle = expand_path(image_bundle)
to_flash = self._bundle_to_images(image_bundle)
to_flash = merge_dicts(to_flash, images or {}, should_normalize=False)
for partition, image_path in to_flash.iteritems():
self.logger.debug('flashing {}'.format(partition))
self._flash_image(self.target, partition, expand_path(image_path))
fastboot_command('reboot')
self.target.connect(timeout=180)
def _validate_image_bundle(self, image_bundle):
if not tarfile.is_tarfile(image_bundle):
raise HostError('File {} is not a tarfile'.format(image_bundle))
with tarfile.open(image_bundle) as tar:
files = [tf.name for tf in tar.getmembers()]
if not any(pf in files for pf in (self.partitions_file_name, '{}/{}'.format(files[0], self.partitions_file_name))):
HostError('Image bundle does not contain the required partition file (see documentation)')
def _bundle_to_images(self, image_bundle):
"""
Extracts the bundle to a temporary location and creates a mapping between the contents of the bundle
and images to be flushed.
"""
self._validate_image_bundle(image_bundle)
extract_dir = tempfile.mkdtemp()
with tarfile.open(image_bundle) as tar:
tar.extractall(path=extract_dir)
files = [tf.name for tf in tar.getmembers()]
if self.partitions_file_name not in files:
extract_dir = os.path.join(extract_dir, files[0])
partition_file = os.path.join(extract_dir, self.partitions_file_name)
return get_mapping(extract_dir, partition_file)
def _flash_image(self, target, partition, image_path):
if not self.prelude_done:
self._fastboot_prelude(target)
fastboot_flash_partition(partition, image_path)
time.sleep(self.delay)
def _fastboot_prelude(self, target):
target.reset(fastboot=True)
time.sleep(self.delay)
self.prelude_done = True
# utility functions
def expand_path(original_path):
path = os.path.abspath(os.path.expanduser(original_path))
if not os.path.exists(path):
raise HostError('{} does not exist.'.format(path))
return path
def get_mapping(base_dir, partition_file):
mapping = {}
with open(partition_file) as pf:
for line in pf:
pair = line.split()
if len(pair) != 2:
HostError('partitions.txt is not properly formated')
image_path = os.path.join(base_dir, pair[1])
if not os.path.isfile(expand_path(image_path)):
HostError('file {} was not found in the bundle or was misplaced'.format(pair[1]))
mapping[pair[0]] = image_path
return mapping

122
devlib/module/biglittle.py Normal file
View File

@@ -0,0 +1,122 @@
from devlib.module import Module
class BigLittleModule(Module):
name = 'bl'
@staticmethod
def probe(target):
return target.big_core is not None
@property
def bigs(self):
return [i for i, c in enumerate(self.target.platform.core_names)
if c == self.target.platform.big_core]
@property
def littles(self):
return [i for i, c in enumerate(self.target.platform.core_names)
if c == self.target.platform.little_core]
@property
def bigs_online(self):
return list(sorted(set(self.bigs).intersection(self.target.list_online_cpus())))
@property
def littles_online(self):
return list(sorted(set(self.littles).intersection(self.target.list_online_cpus())))
# hotplug
def online_all_bigs(self):
self.target.hotplug.online(*self.bigs)
def offline_all_bigs(self):
self.target.hotplug.offline(*self.bigs)
def online_all_littles(self):
self.target.hotplug.online(*self.littles)
def offline_all_littles(self):
self.target.hotplug.offline(*self.littles)
# cpufreq
def list_bigs_frequencies(self):
return self.target.cpufreq.list_frequencies(self.bigs_online[0])
def list_bigs_governors(self):
return self.target.cpufreq.list_governors(self.bigs_online[0])
def list_bigs_governor_tunables(self):
return self.target.cpufreq.list_governor_tunables(self.bigs_online[0])
def list_littles_frequencies(self):
return self.target.cpufreq.list_frequencies(self.littles_online[0])
def list_littles_governors(self):
return self.target.cpufreq.list_governors(self.littles_online[0])
def list_littles_governor_tunables(self):
return self.target.cpufreq.list_governor_tunables(self.littles_online[0])
def get_bigs_governor(self):
return self.target.cpufreq.get_governor(self.bigs_online[0])
def get_bigs_governor_tunables(self):
return self.target.cpufreq.get_governor_tunables(self.bigs_online[0])
def get_bigs_frequency(self):
return self.target.cpufreq.get_frequency(self.bigs_online[0])
def get_bigs_min_frequency(self):
return self.target.cpufreq.get_min_frequency(self.bigs_online[0])
def get_bigs_max_frequency(self):
return self.target.cpufreq.get_max_frequency(self.bigs_online[0])
def get_littles_governor(self):
return self.target.cpufreq.get_governor(self.littles_online[0])
def get_littles_governor_tunables(self):
return self.target.cpufreq.get_governor_tunables(self.littles_online[0])
def get_littles_frequency(self):
return self.target.cpufreq.get_frequency(self.littles_online[0])
def get_littles_min_frequency(self):
return self.target.cpufreq.get_min_frequency(self.littles_online[0])
def get_littles_max_frequency(self):
return self.target.cpufreq.get_max_frequency(self.littles_online[0])
def set_bigs_governor(self, governor, **kwargs):
self.target.cpufreq.set_governor(self.bigs_online[0], governor, **kwargs)
def set_bigs_governor_tunables(self, governor, **kwargs):
self.target.cpufreq.set_governor_tunables(self.bigs_online[0], governor, **kwargs)
def set_bigs_frequency(self, frequency, exact=True):
self.target.cpufreq.set_frequency(self.bigs_online[0], frequency, exact)
def set_bigs_min_frequency(self, frequency, exact=True):
self.target.cpufreq.set_min_frequency(self.bigs_online[0], frequency, exact)
def set_bigs_max_frequency(self, frequency, exact=True):
self.target.cpufreq.set_max_frequency(self.bigs_online[0], frequency, exact)
def set_littles_governor(self, governor, **kwargs):
self.target.cpufreq.set_governor(self.littles_online[0], governor, **kwargs)
def set_littles_governor_tunables(self, governor, **kwargs):
self.target.cpufreq.set_governor_tunables(self.littles_online[0], governor, **kwargs)
def set_littles_frequency(self, frequency, exact=True):
self.target.cpufreq.set_frequency(self.littles_online[0], frequency, exact)
def set_littles_min_frequency(self, frequency, exact=True):
self.target.cpufreq.set_min_frequency(self.littles_online[0], frequency, exact)
def set_littles_max_frequency(self, frequency, exact=True):
self.target.cpufreq.set_max_frequency(self.littles_online[0], frequency, exact)

206
devlib/module/cgroups.py Normal file
View File

@@ -0,0 +1,206 @@
# Copyright 2014-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# pylint: disable=attribute-defined-outside-init
import logging
from collections import namedtuple
from devlib.module import Module
from devlib.exception import TargetError
from devlib.utils.misc import list_to_ranges, isiterable
from devlib.utils.types import boolean
class CgroupController(object):
kind = 'cpuset'
def __new__(cls, arg):
if isinstance(arg, cls):
return arg
else:
return object.__new__(cls, arg)
def __init__(self, mount_name):
self.mount_point = None
self.mount_name = mount_name
self.logger = logging.getLogger(self.kind)
def probe(self, target):
raise NotImplementedError()
def mount(self, device, mount_root):
self.target = device
self.mount_point = device.path.join(mount_root, self.mount_name)
mounted = self.target.list_file_systems()
if self.mount_point in [e.mount_point for e in mounted]:
self.logger.debug('controller is already mounted.')
else:
self.target.execute('mkdir -p {} 2>/dev/null'.format(self.mount_point),
as_root=True)
self.target.execute('mount -t cgroup -o {} {} {}'.format(self.kind,
self.mount_name,
self.mount_point),
as_root=True)
class CpusetGroup(object):
def __init__(self, controller, name, cpus, mems):
self.controller = controller
self.target = controller.target
self.name = name
if name == 'root':
self.directory = controller.mount_point
else:
self.directory = self.target.path.join(controller.mount_point, name)
self.target.execute('mkdir -p {}'.format(self.directory), as_root=True)
self.cpus_file = self.target.path.join(self.directory, 'cpuset.cpus')
self.mems_file = self.target.path.join(self.directory, 'cpuset.mems')
self.tasks_file = self.target.path.join(self.directory, 'tasks')
self.set(cpus, mems)
def set(self, cpus, mems):
if isiterable(cpus):
cpus = list_to_ranges(cpus)
if isiterable(mems):
mems = list_to_ranges(mems)
self.target.write_value(self.cpus_file, cpus)
self.target.write_value(self.mems_file, mems)
def get(self):
cpus = self.target.read_value(self.cpus_file)
mems = self.target.read_value(self.mems_file)
return (cpus, mems)
def get_tasks(self):
task_ids = self.target.read_value(self.tasks_file).split()
return map(int, task_ids)
def add_tasks(self, tasks):
for tid in tasks:
self.add_task(tid)
def add_task(self, tid):
self.target.write_value(self.tasks_file, tid, verify=False)
class CpusetController(CgroupController):
name = 'cpuset'
def __init__(self, *args, **kwargs):
super(CpusetController, self).__init__(*args, **kwargs)
self.groups = {}
def probe(self, target):
return target.config.is_enabled('cpusets')
def mount(self, device, mount_root):
super(CpusetController, self).mount(device, mount_root)
self.create_group('root', self.target.list_online_cpus(), 0)
def create_group(self, name, cpus, mems):
if not hasattr(self, 'target'):
raise RuntimeError('Attempting to create group for unmounted controller {}'.format(self.kind))
if name in self.groups:
raise ValueError('Group {} already exists'.format(name))
self.groups[name] = CpusetGroup(self, name, cpus, mems)
def move_tasks(self, source, dest):
try:
source_group = self.groups[source]
dest_group = self.groups[dest]
command = 'for task in $(cat {}); do echo $task>{}; done'
self.target.execute(command.format(source_group.tasks_file, dest_group.tasks_file),
# this will always fail as some of the tasks
# are kthreads that cannot be migrated, but we
# don't care about those, so don't check exit
# code.
check_exit_code=False, as_root=True)
except KeyError as e:
raise ValueError('Unkown group: {}'.format(e))
def move_all_tasks_to(self, target_group):
for group in self.groups:
if group != target_group:
self.move_tasks(group, target_group)
def __getattr__(self, name):
try:
return self.groups[name]
except KeyError:
raise AttributeError(name)
CgroupSubsystemEntry = namedtuple('CgroupSubsystemEntry', 'name hierarchy num_cgroups enabled')
class CgroupsModule(Module):
name = 'cgroups'
controller_cls = [
CpusetController,
]
cgroup_root = '/sys/fs/cgroup'
@staticmethod
def probe(target):
return target.config.has('cgroups') and target.is_rooted
def __init__(self, target):
super(CgroupsModule, self).__init__(target)
mounted = self.target.list_file_systems()
if self.cgroup_root not in [e.mount_point for e in mounted]:
self.target.execute('mount -t tmpfs {} {}'.format('cgroup_root', self.cgroup_root),
as_root=True)
else:
self.logger.debug('cgroup_root already mounted at {}'.format(self.cgroup_root))
self.controllers = []
for cls in self.controller_cls:
controller = cls('devlib_{}'.format(cls.name))
if controller.probe(self.target):
if controller.mount_name in [e.device for e in mounted]:
self.logger.debug('controller {} is already mounted.'.format(controller.kind))
else:
try:
controller.mount(self.target, self.cgroup_root)
except TargetError:
message = 'cgroups {} controller is not supported by the target'
raise TargetError(message.format(controller.kind))
def list_subsystems(self):
subsystems = []
for line in self.target.execute('cat /proc/cgroups').split('\n')[1:]:
line = line.strip()
if not line or line.startswith('#'):
continue
name, hierarchy, num_cgroups, enabled = line.split()
subsystems.append(CgroupSubsystemEntry(name,
int(hierarchy),
int(num_cgroups),
boolean(enabled)))
return subsystems
def get_cgroup_controller(self, kind):
for controller in self.controllers:
if controller.kind == kind:
return controller
raise ValueError(kind)

63
devlib/module/cooling.py Normal file
View File

@@ -0,0 +1,63 @@
# Copyright 2014-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from devlib.module import Module
from devlib.utils.serial_port import open_serial_connection
class MbedFanActiveCoolingModule(Module):
name = 'mbed-fan'
timeout = 30
@staticmethod
def probe(target):
return True
def __init__(self, target, port='/dev/ttyACM0', baud=115200, fan_pin=0):
super(MbedFanActiveCoolingModule, self).__init__(target)
self.port = port
self.baud = baud
self.fan_pin = fan_pin
def start(self):
with open_serial_connection(timeout=self.timeout,
port=self.port,
baudrate=self.baud) as target:
target.sendline('motor_{}_1'.format(self.fan_pin))
def stop(self):
with open_serial_connection(timeout=self.timeout,
port=self.port,
baudrate=self.baud) as target:
target.sendline('motor_{}_0'.format(self.fan_pin))
class OdroidXU3ctiveCoolingModule(Module):
name = 'odroidxu3-fan'
@staticmethod
def probe(target):
return target.file_exists('/sys/devices/odroid_fan.15/fan_mode')
def start(self):
self.target.write_value('/sys/devices/odroid_fan.15/fan_mode', 0, verify=False)
self.target.write_value('/sys/devices/odroid_fan.15/pwm_duty', 255, verify=False)
def stop(self):
self.target.write_value('/sys/devices/odroid_fan.15/fan_mode', 0, verify=False)
self.target.write_value('/sys/devices/odroid_fan.15/pwm_duty', 1, verify=False)

315
devlib/module/cpufreq.py Normal file
View File

@@ -0,0 +1,315 @@
# Copyright 2014-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from devlib.module import Module
from devlib.exception import TargetError
from devlib.utils.misc import memoized
# a dict of governor name and a list of it tunables that can't be read
WRITE_ONLY_TUNABLES = {
'interactive': ['boostpulse']
}
class CpufreqModule(Module):
name = 'cpufreq'
@staticmethod
def probe(target):
path = '/sys/devices/system/cpu/cpufreq'
return target.file_exists(path)
def __init__(self, target):
super(CpufreqModule, self).__init__(target)
self._governor_tunables = {}
@memoized
def list_governors(self, cpu):
"""Returns a list of governors supported by the cpu."""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
sysfile = '/sys/devices/system/cpu/{}/cpufreq/scaling_available_governors'.format(cpu)
output = self.target.read_value(sysfile)
return output.strip().split()
def get_governor(self, cpu):
"""Returns the governor currently set for the specified CPU."""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
sysfile = '/sys/devices/system/cpu/{}/cpufreq/scaling_governor'.format(cpu)
return self.target.read_value(sysfile)
def set_governor(self, cpu, governor, **kwargs):
"""
Set the governor for the specified CPU.
See https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt
:param cpu: The CPU for which the governor is to be set. This must be
the full name as it appears in sysfs, e.g. "cpu0".
:param governor: The name of the governor to be used. This must be
supported by the specific device.
Additional keyword arguments can be used to specify governor tunables for
governors that support them.
:note: On big.LITTLE all cores in a cluster must be using the same governor.
Setting the governor on any core in a cluster will also set it on all
other cores in that cluster.
:raises: TargetError if governor is not supported by the CPU, or if,
for some reason, the governor could not be set.
"""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
supported = self.list_governors(cpu)
if governor not in supported:
raise TargetError('Governor {} not supported for cpu {}'.format(governor, cpu))
sysfile = '/sys/devices/system/cpu/{}/cpufreq/scaling_governor'.format(cpu)
self.target.write_value(sysfile, governor)
self.set_governor_tunables(cpu, governor, **kwargs)
def list_governor_tunables(self, cpu):
"""Returns a list of tunables available for the governor on the specified CPU."""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
governor = self.get_governor(cpu)
if governor not in self._governor_tunables:
try:
tunables_path = '/sys/devices/system/cpu/{}/cpufreq/{}'.format(cpu, governor)
self._governor_tunables[governor] = self.target.list_directory(tunables_path)
except TargetError: # probably an older kernel
try:
tunables_path = '/sys/devices/system/cpu/cpufreq/{}'.format(governor)
self._governor_tunables[governor] = self.target.list_directory(tunables_path)
except TargetError: # governor does not support tunables
self._governor_tunables[governor] = []
return self._governor_tunables[governor]
def get_governor_tunables(self, cpu):
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
governor = self.get_governor(cpu)
tunables = {}
for tunable in self.list_governor_tunables(cpu):
if tunable not in WRITE_ONLY_TUNABLES.get(governor, []):
try:
path = '/sys/devices/system/cpu/{}/cpufreq/{}/{}'.format(cpu, governor, tunable)
tunables[tunable] = self.target.read_value(path)
except TargetError: # May be an older kernel
path = '/sys/devices/system/cpu/cpufreq/{}/{}'.format(governor, tunable)
tunables[tunable] = self.target.read_value(path)
return tunables
def set_governor_tunables(self, cpu, governor=None, **kwargs):
"""
Set tunables for the specified governor. Tunables should be specified as
keyword arguments. Which tunables and values are valid depends on the
governor.
:param cpu: The cpu for which the governor will be set. This must be the
full cpu name as it appears in sysfs, e.g. ``cpu0``.
:param governor: The name of the governor. Must be all lower case.
The rest should be keyword parameters mapping tunable name onto the value to
be set for it.
:raises: TargetError if governor specified is not a valid governor name, or if
a tunable specified is not valid for the governor, or if could not set
tunable.
"""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
if governor is None:
governor = self.get_governor(cpu)
valid_tunables = self.list_governor_tunables(cpu)
for tunable, value in kwargs.iteritems():
if tunable in valid_tunables:
try:
path = '/sys/devices/system/cpu/{}/cpufreq/{}/{}'.format(cpu, governor, tunable)
self.target.write_value(path, value)
except TargetError: # May be an older kernel
path = '/sys/devices/system/cpu/cpufreq/{}/{}'.format(governor, tunable)
self.target.write_value(path, value)
else:
message = 'Unexpected tunable {} for governor {} on {}.\n'.format(tunable, governor, cpu)
message += 'Available tunables are: {}'.format(valid_tunables)
raise TargetError(message)
@memoized
def list_frequencies(self, cpu):
"""Returns a list of frequencies supported by the cpu or an empty list
if not could be found."""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
try:
cmd = 'cat /sys/devices/system/cpu/{}/cpufreq/scaling_available_frequencies'.format(cpu)
output = self.target.execute(cmd)
available_frequencies = map(int, output.strip().split()) # pylint: disable=E1103
except TargetError:
# On some devices scaling_frequencies is not generated.
# http://adrynalyne-teachtofish.blogspot.co.uk/2011/11/how-to-enable-scalingavailablefrequenci.html
# Fall back to parsing stats/time_in_state
cmd = 'cat /sys/devices/system/cpu/{}/cpufreq/stats/time_in_state'.format(cpu)
out_iter = iter(self.target.execute(cmd).strip().split())
available_frequencies = map(int, reversed([f for f, _ in zip(out_iter, out_iter)]))
return available_frequencies
def get_min_frequency(self, cpu):
"""
Returns the min frequency currently set for the specified CPU.
Warning, this method does not check if the cpu is online or not. It will
try to read the minimum frequency and the following exception will be
raised ::
:raises: TargetError if for some reason the frequency could not be read.
"""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
sysfile = '/sys/devices/system/cpu/{}/cpufreq/scaling_min_freq'.format(cpu)
return self.target.read_int(sysfile)
def set_min_frequency(self, cpu, frequency, exact=True):
"""
Set's the minimum value for CPU frequency. Actual frequency will
depend on the Governor used and may vary during execution. The value should be
either an int or a string representing an integer. The Value must also be
supported by the device. The available frequencies can be obtained by calling
get_frequencies() or examining
/sys/devices/system/cpu/cpuX/cpufreq/scaling_frequencies
on the device.
:raises: TargetError if the frequency is not supported by the CPU, or if, for
some reason, frequency could not be set.
:raises: ValueError if ``frequency`` is not an integer.
"""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
available_frequencies = self.list_frequencies(cpu)
try:
value = int(frequency)
if exact and available_frequencies and value not in available_frequencies:
raise TargetError('Can\'t set {} frequency to {}\nmust be in {}'.format(cpu,
value,
available_frequencies))
sysfile = '/sys/devices/system/cpu/{}/cpufreq/scaling_min_freq'.format(cpu)
self.target.write_value(sysfile, value)
except ValueError:
raise ValueError('Frequency must be an integer; got: "{}"'.format(frequency))
def get_frequency(self, cpu):
"""
Returns the current frequency currently set for the specified CPU.
Warning, this method does not check if the cpu is online or not. It will
try to read the current frequency and the following exception will be
raised ::
:raises: TargetError if for some reason the frequency could not be read.
"""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
sysfile = '/sys/devices/system/cpu/{}/cpufreq/scaling_cur_freq'.format(cpu)
return self.target.read_int(sysfile)
def set_frequency(self, cpu, frequency, exact=True):
"""
Set's the minimum value for CPU frequency. Actual frequency will
depend on the Governor used and may vary during execution. The value should be
either an int or a string representing an integer.
If ``exact`` flag is set (the default), the Value must also be supported by
the device. The available frequencies can be obtained by calling
get_frequencies() or examining
/sys/devices/system/cpu/cpuX/cpufreq/scaling_frequencies
on the device (if it exists).
:raises: TargetError if the frequency is not supported by the CPU, or if, for
some reason, frequency could not be set.
:raises: ValueError if ``frequency`` is not an integer.
"""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
try:
value = int(frequency)
if exact:
available_frequencies = self.list_frequencies(cpu)
if available_frequencies and value not in available_frequencies:
raise TargetError('Can\'t set {} frequency to {}\nmust be in {}'.format(cpu,
value,
available_frequencies))
if self.get_governor(cpu) != 'userspace':
raise TargetError('Can\'t set {} frequency; governor must be "userspace"'.format(cpu))
sysfile = '/sys/devices/system/cpu/{}/cpufreq/scaling_setspeed'.format(cpu)
self.target.write_value(sysfile, value, verify=False)
except ValueError:
raise ValueError('Frequency must be an integer; got: "{}"'.format(frequency))
def get_max_frequency(self, cpu):
"""
Returns the max frequency currently set for the specified CPU.
Warning, this method does not check if the cpu is online or not. It will
try to read the maximum frequency and the following exception will be
raised ::
:raises: TargetError if for some reason the frequency could not be read.
"""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
sysfile = '/sys/devices/system/cpu/{}/cpufreq/scaling_max_freq'.format(cpu)
return self.target.read_int(sysfile)
def set_max_frequency(self, cpu, frequency, exact=True):
"""
Set's the minimum value for CPU frequency. Actual frequency will
depend on the Governor used and may vary during execution. The value should be
either an int or a string representing an integer. The Value must also be
supported by the device. The available frequencies can be obtained by calling
get_frequencies() or examining
/sys/devices/system/cpu/cpuX/cpufreq/scaling_frequencies
on the device.
:raises: TargetError if the frequency is not supported by the CPU, or if, for
some reason, frequency could not be set.
:raises: ValueError if ``frequency`` is not an integer.
"""
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
available_frequencies = self.list_frequencies(cpu)
try:
value = int(frequency)
if exact and available_frequencies and value not in available_frequencies:
raise TargetError('Can\'t set {} frequency to {}\nmust be in {}'.format(cpu,
value,
available_frequencies))
sysfile = '/sys/devices/system/cpu/{}/cpufreq/scaling_max_freq'.format(cpu)
self.target.write_value(sysfile, value)
except ValueError:
raise ValueError('Frequency must be an integer; got: "{}"'.format(frequency))

138
devlib/module/cpuidle.py Normal file
View File

@@ -0,0 +1,138 @@
# Copyright 2014-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# pylint: disable=attribute-defined-outside-init
from devlib.module import Module
from devlib.utils.misc import memoized
from devlib.utils.types import integer, boolean
class CpuidleState(object):
@property
def usage(self):
return integer(self.get('usage'))
@property
def time(self):
return integer(self.get('time'))
@property
def is_enabled(self):
return not boolean(self.get('disable'))
@property
def ordinal(self):
i = len(self.id)
while self.id[i - 1].isdigit():
i -= 1
if not i:
raise ValueError('invalid idle state name: "{}"'.format(self.id))
return int(self.id[i:])
def __init__(self, target, index, path):
self.target = target
self.index = index
self.path = path
self.id = self.target.path.basename(self.path)
self.cpu = self.target.path.basename(self.target.path.dirname(path))
self.desc = self.get('desc')
self.name = self.get('name')
self.latency = self.get('latency')
self.power = self.get('power')
def enable(self):
self.set('disable', 0)
def disable(self):
self.set('disable', 1)
def get(self, prop):
property_path = self.target.path.join(self.path, prop)
return self.target.read_value(property_path)
def set(self, prop, value):
property_path = self.target.path.join(self.path, prop)
self.target.write_value(property_path, value)
def __eq__(self, other):
if isinstance(other, CpuidleState):
return (self.name == other.name) and (self.desc == other.desc)
elif isinstance(other, basestring):
return (self.name == other) or (self.desc == other)
else:
return False
def __ne__(self, other):
return not self.__eq__(other)
def __str__(self):
return 'CpuidleState({}, {})'.format(self.name, self.desc)
__repr__ = __str__
class Cpuidle(Module):
name = 'cpuidle'
root_path = '/sys/devices/system/cpu/cpuidle'
@staticmethod
def probe(target):
return target.file_exists(Cpuidle.root_path)
def get_driver(self):
return self.target.read_value(self.target.path.join(self.root_path, 'current_driver'))
def get_governor(self):
return self.target.read_value(self.target.path.join(self.root_path, 'current_governor_ro'))
@memoized
def get_states(self, cpu=0):
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
states_dir = self.target.path.join(self.target.path.dirname(self.root_path), cpu, 'cpuidle')
idle_states = []
for state in self.target.list_directory(states_dir):
if state.startswith('state'):
index = int(state[5:])
idle_states.append(CpuidleState(self.target, index, self.target.path.join(states_dir, state)))
return idle_states
def get_state(self, state, cpu=0):
if isinstance(state, int):
try:
self.get_states(cpu)[state].enable()
except IndexError:
raise ValueError('Cpuidle state {} does not exist'.format(state))
else: # assume string-like
for s in self.get_states(cpu):
if state in [s.id, s.name, s.desc]:
return s
raise ValueError('Cpuidle state {} does not exist'.format(state))
def enable(self, state, cpu=0):
self.get_state(state, cpu).enable()
def disable(self, state, cpu=0):
self.get_state(state, cpu).disable()
def enable_all(self, cpu=0):
for state in self.get_states(cpu):
state.enable()
def disable_all(self, cpu=0):
for state in self.get_states(cpu):
state.disable()

35
devlib/module/hotplug.py Normal file
View File

@@ -0,0 +1,35 @@
from devlib.module import Module
class HotplugModule(Module):
name = 'hotplug'
base_path = '/sys/devices/system/cpu'
@classmethod
def probe(cls, target): # pylint: disable=arguments-differ
path = cls._cpu_path(target, 0)
return target.file_exists(path) and target.is_rooted
@classmethod
def _cpu_path(cls, target, cpu):
if isinstance(cpu, int):
cpu = 'cpu{}'.format(cpu)
return target.path.join(cls.base_path, cpu, 'online')
def online_all(self):
self.online(*range(self.target.number_of_cpus))
def online(self, *args):
for cpu in args:
self.hotplug(cpu, online=True)
def offline(self, *args):
for cpu in args:
self.hotplug(cpu, online=False)
def hotplug(self, cpu, online):
path = self._cpu_path(self.target, cpu)
value = 1 if online else 0
self.target.write_value(path, value)

140
devlib/module/hwmon.py Normal file
View File

@@ -0,0 +1,140 @@
# Copyright 2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import re
from collections import defaultdict
from devlib.module import Module
from devlib.utils.types import integer
HWMON_ROOT = '/sys/class/hwmon'
HWMON_FILE_REGEX = re.compile(r'(?P<kind>\w+?)(?P<number>\d+)_(?P<item>\w+)')
class HwmonSensor(object):
def __init__(self, device, path, kind, number):
self.device = device
self.path = path
self.kind = kind
self.number = number
self.target = self.device.target
self.name = '{}/{}{}'.format(self.device.name, self.kind, self.number)
self.label = self.name
self.items = set()
def add(self, item):
self.items.add(item)
if item == 'label':
self.label = self.get('label')
def get(self, item):
path = self.get_file(item)
value = self.target.read_value(path)
try:
return integer(value)
except (TypeError, ValueError):
return value
def set(self, item, value):
path = self.get_file(item)
self.target.write_value(path, value)
def get_file(self, item):
if item not in self.items:
raise ValueError('item "{}" does not exist for {}'.format(item, self.name))
filename = '{}{}_{}'.format(self.kind, self.number, item)
return self.target.path.join(self.path, filename)
def __str__(self):
if self.name != self.label:
text = 'HS({}, {})'.format(self.name, self.label)
else:
text = 'HS({})'.format(self.name)
return text
__repr__ = __str__
class HwmonDevice(object):
@property
def sensors(self):
all_sensors = []
for sensors_of_kind in self._sensors.itervalues():
all_sensors.extend(sensors_of_kind.values())
return all_sensors
def __init__(self, target, path):
self.target = target
self.path = path
self.name = self.target.read_value(self.target.path.join(self.path, 'name'))
self._sensors = defaultdict(dict)
path = self.path
if not path.endswith(self.target.path.sep):
path += self.target.path.sep
for entry in self.target.list_directory(path):
match = HWMON_FILE_REGEX.search(entry)
if match:
kind = match.group('kind')
number = int(match.group('number'))
item = match.group('item')
if number not in self._sensors[kind]:
sensor = HwmonSensor(self, self.path, kind, number)
self._sensors[kind][number] = sensor
self._sensors[kind][number].add(item)
def get(self, kind, number=None):
if number is None:
return [s for _, s in sorted(self._sensors[kind].iteritems(),
key=lambda x: x[0])]
else:
return self._sensors[kind].get(number)
def __str__(self):
return 'HD({})'.format(self.name)
__repr__ = __str__
class HwmonModule(Module):
name = 'hwmon'
@staticmethod
def probe(target):
return target.file_exists(HWMON_ROOT)
@property
def sensors(self):
all_sensors = []
for device in self.devices:
all_sensors.extend(device.sensors)
return all_sensors
def __init__(self, target):
super(HwmonModule, self).__init__(target)
self.root = HWMON_ROOT
self.devices = []
self.scan()
def scan(self):
for entry in self.target.list_directory(self.root):
if entry.startswith('hwmon'):
entry_path = self.target.path.join(self.root, entry)
if self.target.file_exists(self.target.path.join(entry_path, 'name')):
device = HwmonDevice(self.target, entry_path)
self.devices.append(device)

386
devlib/module/vexpress.py Normal file
View File

@@ -0,0 +1,386 @@
#
# Copyright 2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import os
import time
import tarfile
import shutil
from devlib.module import HardRestModule, BootModule, FlashModule
from devlib.exception import TargetError, HostError
from devlib.utils.serial_port import open_serial_connection, pulse_dtr, write_characters
from devlib.utils.uefi import UefiMenu, UefiConfig
from devlib.utils.uboot import UbootMenu
AUTOSTART_MESSAGE = 'Press Enter to stop auto boot...'
POWERUP_MESSAGE = 'Powering up system...'
DEFAULT_MCC_PROMPT = 'Cmd>'
class VexpressDtrHardReset(HardRestModule):
name = 'vexpress-dtr'
stage = 'early'
@staticmethod
def probe(target):
return True
def __init__(self, target, port='/dev/ttyS0', baudrate=115200,
mcc_prompt=DEFAULT_MCC_PROMPT, timeout=300):
super(VexpressDtrHardReset, self).__init__(target)
self.port = port
self.baudrate = baudrate
self.mcc_prompt = mcc_prompt
self.timeout = timeout
def __call__(self):
try:
if self.target.is_connected:
self.target.execute('sync')
except TargetError:
pass
with open_serial_connection(port=self.port,
baudrate=self.baudrate,
timeout=self.timeout,
init_dtr=0,
get_conn=True) as (_, conn):
pulse_dtr(conn, state=True, duration=0.1) # TRM specifies a pulse of >=100ms
class VexpressReboottxtHardReset(HardRestModule):
name = 'vexpress-reboottxt'
stage = 'early'
@staticmethod
def probe(target):
return True
def __init__(self, target,
port='/dev/ttyS0', baudrate=115200,
path='/media/VEMSD',
mcc_prompt=DEFAULT_MCC_PROMPT, timeout=30, short_delay=1):
super(VexpressReboottxtHardReset, self).__init__(target)
self.port = port
self.baudrate = baudrate
self.path = path
self.mcc_prompt = mcc_prompt
self.timeout = timeout
self.short_delay = short_delay
self.filepath = os.path.join(path, 'reboot.txt')
def __call__(self):
try:
if self.target.is_connected:
self.target.execute('sync')
except TargetError:
pass
if not os.path.exists(self.path):
self.logger.debug('{} does not exisit; attempting to mount...'.format(self.path))
with open_serial_connection(port=self.port,
baudrate=self.baudrate,
timeout=self.timeout,
init_dtr=0) as tty:
wait_for_vemsd(self.path, tty, self.mcc_prompt, self.short_delay)
with open(self.filepath, 'w'):
pass
class VexpressBootModule(BootModule):
stage = 'early'
@staticmethod
def probe(target):
return True
def __init__(self, target, uefi_entry=None,
port='/dev/ttyS0', baudrate=115200,
mcc_prompt=DEFAULT_MCC_PROMPT,
timeout=120, short_delay=1):
super(VexpressBootModule, self).__init__(target)
self.port = port
self.baudrate = baudrate
self.uefi_entry = uefi_entry
self.mcc_prompt = mcc_prompt
self.timeout = timeout
self.short_delay = short_delay
def __call__(self):
with open_serial_connection(port=self.port,
baudrate=self.baudrate,
timeout=self.timeout,
init_dtr=0) as tty:
self.get_through_early_boot(tty)
self.perform_boot_sequence(tty)
self.wait_for_android_prompt(tty)
def perform_boot_sequence(self, tty):
raise NotImplementedError()
def get_through_early_boot(self, tty):
self.logger.debug('Establishing initial state...')
tty.sendline('')
i = tty.expect([AUTOSTART_MESSAGE, POWERUP_MESSAGE, self.mcc_prompt])
if i == 2:
self.logger.debug('Saw MCC prompt.')
time.sleep(self.short_delay)
tty.sendline('reboot')
elif i == 1:
self.logger.debug('Saw powering up message (assuming soft reboot).')
else:
self.logger.debug('Saw auto boot message.')
tty.sendline('')
time.sleep(self.short_delay)
tty.sendline('reboot')
def get_uefi_menu(self, tty):
menu = UefiMenu(tty)
self.logger.debug('Waiting for UEFI menu...')
menu.wait(timeout=self.timeout)
return menu
def wait_for_android_prompt(self, tty):
self.logger.debug('Waiting for the Android prompt.')
tty.expect(self.target.shell_prompt, timeout=self.timeout)
# This delay is needed to allow the platform some time to finish
# initilizing; querying the ip address too early from connect() may
# result in a bogus address being assigned to eth0.
time.sleep(5)
class VexpressUefiBoot(VexpressBootModule):
name = 'vexpress-uefi'
def __init__(self, target, uefi_entry,
image, fdt, bootargs, initrd,
*args, **kwargs):
super(VexpressUefiBoot, self).__init__(target, uefi_entry=uefi_entry,
*args, **kwargs)
self.uefi_config = self._create_config(image, fdt, bootargs, initrd)
def perform_boot_sequence(self, tty):
menu = self.get_uefi_menu(tty)
try:
menu.select(self.uefi_entry)
except LookupError:
self.logger.debug('{} UEFI entry not found.'.format(self.uefi_entry))
self.logger.debug('Attempting to create one using default flasher configuration.')
menu.create_entry(self.uefi_entry, self.uefi_config)
menu.select(self.uefi_entry)
def _create_config(self, image, fdt, bootargs, initrd): # pylint: disable=R0201
config_dict = {
'image_name': image,
'image_args': bootargs,
'initrd': initrd,
}
if fdt:
config_dict['fdt_support'] = True
config_dict['fdt_path'] = fdt
else:
config_dict['fdt_support'] = False
return UefiConfig(config_dict)
class VexpressUefiShellBoot(VexpressBootModule):
name = 'vexpress-uefi-shell'
def __init__(self, target, uefi_entry='^Shell$',
efi_shell_prompt='Shell>',
image='kernel', bootargs=None,
*args, **kwargs):
super(VexpressUefiShellBoot, self).__init__(target, uefi_entry=uefi_entry,
*args, **kwargs)
self.efi_shell_prompt = efi_shell_prompt
self.image = image
self.bootargs = bootargs
def perform_boot_sequence(self, tty):
menu = self.get_uefi_menu(tty)
try:
menu.select(self.uefi_entry)
except LookupError:
raise TargetError('Did not see "{}" UEFI entry.'.format(self.uefi_entry))
tty.expect(self.efi_shell_prompt, timeout=self.timeout)
if self.bootargs:
tty.sendline('') # stop default boot
time.sleep(self.short_delay)
efi_shell_command = '{} {}'.format(self.image, self.bootargs)
self.logger.debug(efi_shell_command)
write_characters(tty, efi_shell_command)
tty.sendline('\r\n')
class VexpressUBoot(VexpressBootModule):
name = 'vexpress-u-boot'
def __init__(self, target, env=None,
*args, **kwargs):
super(VexpressUBoot, self).__init__(target, *args, **kwargs)
self.env = env
def perform_boot_sequence(self, tty):
if self.env is None:
return # Will boot automatically
menu = UbootMenu(tty)
self.logger.debug('Waiting for U-Boot prompt...')
menu.open(timeout=120)
for var, value in self.env.iteritems():
menu.setenv(var, value)
menu.boot()
class VexpressBootmon(VexpressBootModule):
name = 'vexpress-bootmon'
def __init__(self, target,
image, fdt, initrd, bootargs,
uses_bootscript=False,
bootmon_prompt='>',
*args, **kwargs):
super(VexpressBootmon, self).__init__(target, *args, **kwargs)
self.image = image
self.fdt = fdt
self.initrd = initrd
self.bootargs = bootargs
self.uses_bootscript = uses_bootscript
self.bootmon_prompt = bootmon_prompt
def perform_boot_sequence(self, tty):
if self.uses_bootscript:
return # Will boot automatically
time.sleep(self.short_delay)
tty.expect(self.bootmon_prompt, timeout=self.timeout)
with open_serial_connection(port=self.port,
baudrate=self.baudrate,
timeout=self.timeout,
init_dtr=0) as tty:
write_characters(tty, 'fl linux fdt {}'.format(self.fdt))
write_characters(tty, 'fl linux initrd {}'.format(self.initrd))
write_characters(tty, 'fl linux boot {} {}'.format(self.image,
self.bootargs))
class VersatileExpressFlashModule(FlashModule):
name = 'vexpress-vemsd'
description = """
Enables flashing of kernels and firmware to ARM Versatile Express devices.
This modules enables flashing of image bundles or individual images to ARM
Versatile Express-based devices (e.g. JUNO) via host-mounted MicroSD on the
board.
The bundle, if specified, must reflect the directory structure of the MicroSD
and will be extracted directly into the location it is mounted on the host. The
images, if specified, must be a dict mapping the absolute path of the image on
the host to the destination path within the board's MicroSD; the destination path
may be either absolute, or relative to the MicroSD mount location.
"""
stage = 'early'
@staticmethod
def probe(target):
if not target.has('hard_reset'):
return False
return True
def __init__(self, target, vemsd_mount, mcc_prompt=DEFAULT_MCC_PROMPT, timeout=30, short_delay=1):
super(VersatileExpressFlashModule, self).__init__(target)
self.vemsd_mount = vemsd_mount
self.mcc_prompt = mcc_prompt
self.timeout = timeout
self.short_delay = short_delay
def __call__(self, image_bundle=None, images=None, bootargs=None):
self.target.hard_reset()
with open_serial_connection(port=self.target.platform.serial_port,
baudrate=self.target.platform.baudrate,
timeout=self.timeout,
init_dtr=0) as tty:
i = tty.expect([self.mcc_prompt, AUTOSTART_MESSAGE])
if i:
tty.sendline('')
wait_for_vemsd(self.vemsd_mount, tty, self.mcc_prompt, self.short_delay)
try:
if image_bundle:
self._deploy_image_bundle(image_bundle)
if images:
self._overlay_images(images)
os.system('sync')
except (IOError, OSError), e:
msg = 'Could not deploy images to {}; got: {}'
raise TargetError(msg.format(self.vemsd_mount, e))
self.target.boot()
self.target.connect(timeout=30)
def _deploy_image_bundle(self, bundle):
self.logger.debug('Validating {}'.format(bundle))
validate_image_bundle(bundle)
self.logger.debug('Extracting {} into {}...'.format(bundle, self.vemsd_mount))
with tarfile.open(bundle) as tar:
tar.extractall(self.vemsd_mount)
def _overlay_images(self, images):
for dest, src in images.iteritems():
dest = os.path.join(self.vemsd_mount, dest)
self.logger.debug('Copying {} to {}'.format(src, dest))
shutil.copy(src, dest)
# utility functions
def validate_image_bundle(bundle):
if not tarfile.is_tarfile(bundle):
raise HostError('Image bundle {} does not appear to be a valid TAR file.'.format(bundle))
with tarfile.open(bundle) as tar:
try:
tar.getmember('config.txt')
except KeyError:
try:
tar.getmember('./config.txt')
except KeyError:
msg = 'Tarball {} does not appear to be a valid image bundle (did not see config.txt).'
raise HostError(msg.format(bundle))
def wait_for_vemsd(vemsd_mount, tty, mcc_prompt=DEFAULT_MCC_PROMPT, short_delay=1, retries=3):
attempts = 1 + retries
path = os.path.join(vemsd_mount, 'config.txt')
if os.path.exists(path):
return
for _ in xrange(attempts):
tty.sendline('') # clear any garbage
tty.expect(mcc_prompt, timeout=short_delay)
tty.sendline('usb_on')
time.sleep(short_delay * 3)
if os.path.exists(path):
return
raise TargetError('Could not mount {}'.format(vemsd_mount))

81
devlib/platform/__init__.py Normal file
View File

@@ -0,0 +1,81 @@
import logging
class Platform(object):
@property
def number_of_clusters(self):
return len(set(self.core_clusters))
def __init__(self,
name=None,
core_names=None,
core_clusters=None,
big_core=None,
model=None,
modules=None,
):
self.name = name
self.core_names = core_names or []
self.core_clusters = core_clusters or []
self.big_core = big_core
self.little_core = None
self.model = model
self.modules = modules or []
self.logger = logging.getLogger(self.name)
if not self.core_clusters and self.core_names:
self._set_core_clusters_from_core_names()
self._validate()
def init_target_connection(self, target):
# May be ovewritten by subclasses to provide target-specific
# connection initialisation.
pass
def update_from_target(self, target):
if not self.core_names:
self.core_names = target.cpuinfo.cpu_names
self._set_core_clusters_from_core_names()
if not self.big_core and self.number_of_clusters == 2:
big_idx = self.core_clusters.index(max(self.core_clusters))
self.big_core = self.core_names[big_idx]
if not self.core_clusters and self.core_names:
self._set_core_clusters_from_core_names()
if not self.model:
self._set_model_from_target(target)
if not self.name:
self.name = self.model
self._validate()
def _set_core_clusters_from_core_names(self):
self.core_clusters = []
clusters = []
for cn in self.core_names:
if cn not in clusters:
clusters.append(cn)
self.core_clusters.append(clusters.index(cn))
def _set_model_from_target(self, target):
if target.os == 'android':
self.model = target.getprop('ro.product.model')
elif target.is_rooted:
try:
self.model = target.execute('dmidecode -s system-version',
as_root=True).strip()
except Exception: # pylint: disable=broad-except
pass # this is best-effort
def _validate(self):
if len(self.core_names) != len(self.core_clusters):
raise ValueError('core_names and core_clusters are of different lengths.')
if self.big_core and self.number_of_clusters != 2:
raise ValueError('attempting to set big_core on non-big.LITTLE device. '
'(number of clusters is not 2)')
if self.big_core and self.big_core not in self.core_names:
message = 'Invalid big_core value "{}"; must be in [{}]'
raise ValueError(message.format(self.big_core,
', '.join(set(self.core_names))))
if self.big_core:
little_idx = self.core_clusters.index(min(self.core_clusters))
self.little_core = self.core_names[little_idx]

280
devlib/platform/arm.py Normal file
View File

@@ -0,0 +1,280 @@
# Copyright 2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import division
import os
import tempfile
import csv
import time
import pexpect
from devlib.platform import Platform
from devlib.instrument import Instrument, InstrumentChannel, MeasurementsCsv, CONTINUOUS
from devlib.exception import TargetError, HostError
from devlib.host import PACKAGE_BIN_DIRECTORY
from devlib.utils.serial_port import open_serial_connection
class VersatileExpressPlatform(Platform):
def __init__(self, name, # pylint: disable=too-many-locals
core_names=None,
core_clusters=None,
big_core=None,
modules=None,
# serial settings
serial_port='/dev/ttyS0',
baudrate=115200,
# VExpress MicroSD mount point
vemsd_mount=None,
# supported: dtr, reboottxt
hard_reset_method=None,
# supported: uefi, uefi-shell, u-boot, bootmon
bootloader=None,
# supported: vemsd
flash_method='vemsd',
image=None,
fdt=None,
initrd=None,
bootargs=None,
uefi_entry=None, # only used if bootloader is "uefi"
ready_timeout=60,
):
super(VersatileExpressPlatform, self).__init__(name,
core_names,
core_clusters,
big_core,
modules)
self.serial_port = serial_port
self.baudrate = baudrate
self.vemsd_mount = vemsd_mount
self.image = image
self.fdt = fdt
self.initrd = initrd
self.bootargs = bootargs
self.uefi_entry = uefi_entry
self.ready_timeout = ready_timeout
self.bootloader = None
self.hard_reset_method = None
self._set_bootloader(bootloader)
self._set_hard_reset_method(hard_reset_method)
self._set_flash_method(flash_method)
def init_target_connection(self, target):
if target.os == 'android':
self._init_android_target(target)
else:
self._init_linux_target(target)
def _init_android_target(self, target):
if target.connection_settings.get('device') is None:
addr = self._get_target_ip_address(target)
target.connection_settings['device'] = addr + ':5555'
def _init_linux_target(self, target):
if target.connection_settings.get('host') is None:
addr = self._get_target_ip_address(target)
target.connection_settings['host'] = addr
def _get_target_ip_address(self, target):
with open_serial_connection(port=self.serial_port,
baudrate=self.baudrate,
timeout=30,
init_dtr=0) as tty:
tty.sendline('')
self.logger.debug('Waiting for the Android shell prompt.')
tty.expect(target.shell_prompt)
self.logger.debug('Waiting for IP address...')
wait_start_time = time.time()
while True:
tty.sendline('ip addr list eth0')
time.sleep(1)
try:
tty.expect(r'inet ([1-9]\d*.\d+.\d+.\d+)', timeout=10)
return tty.match.group(1)
except pexpect.TIMEOUT:
pass # We have our own timeout -- see below.
if (time.time() - wait_start_time) > self.ready_timeout:
raise TargetError('Could not acquire IP address.')
def _set_hard_reset_method(self, hard_reset_method):
if hard_reset_method == 'dtr':
self.modules.append({'vexpress-dtr': {'port': self.serial_port,
'baudrate': self.baudrate,
}})
elif hard_reset_method == 'reboottxt':
self.modules.append({'vexpress-reboottxt': {'port': self.serial_port,
'baudrate': self.baudrate,
'path': self.vemsd_mount,
}})
else:
ValueError('Invalid hard_reset_method: {}'.format(hard_reset_method))
def _set_bootloader(self, bootloader):
self.bootloader = bootloader
if self.bootloader == 'uefi':
self.modules.append({'vexpress-uefi': {'port': self.serial_port,
'baudrate': self.baudrate,
'image': self.image,
'fdt': self.fdt,
'initrd': self.initrd,
'bootargs': self.bootargs,
}})
elif self.bootloader == 'uefi-shell':
self.modules.append({'vexpress-uefi-shell': {'port': self.serial_port,
'baudrate': self.baudrate,
'image': self.image,
'bootargs': self.bootargs,
}})
elif self.bootloader == 'u-boot':
self.modules.append({'vexpress-u-boot': {'port': self.serial_port,
'baudrate': self.baudrate,
'env': {'bootargs': self.bootargs},
}})
elif self.bootloader == 'bootmon':
self.modules.append({'vexpress-bootmon': {'port': self.serial_port,
'baudrate': self.baudrate,
'image': self.image,
'fdt': self.fdt,
'initrd': self.initrd,
'bootargs': self.bootargs,
}})
else:
ValueError('Invalid hard_reset_method: {}'.format(bootloader))
def _set_flash_method(self, flash_method):
if flash_method == 'vemsd':
self.modules.append({'vexpress-vemsd': {'vemsd_mount': self.vemsd_mount}})
else:
ValueError('Invalid flash_method: {}'.format(flash_method))
class Juno(VersatileExpressPlatform):
def __init__(self,
vemsd_mount='/media/JUNO',
baudrate=115200,
bootloader='u-boot',
hard_reset_method='dtr',
**kwargs
):
super(Juno, self).__init__('juno',
vemsd_mount=vemsd_mount,
baudrate=baudrate,
bootloader=bootloader,
hard_reset_method=hard_reset_method,
**kwargs)
class TC2(VersatileExpressPlatform):
def __init__(self,
vemsd_mount='/media/VEMSD',
baudrate=38400,
bootloader='bootmon',
hard_reset_method='reboottxt',
**kwargs
):
super(TC2, self).__init__('tc2',
vemsd_mount=vemsd_mount,
baudrate=baudrate,
bootloader=bootloader,
hard_reset_method=hard_reset_method,
**kwargs)
class JunoEnergyInstrument(Instrument):
binname = 'readenergy'
mode = CONTINUOUS
_channels = [
InstrumentChannel('sys_curr', 'sys', 'current'),
InstrumentChannel('a57_curr', 'a57', 'current'),
InstrumentChannel('a53_curr', 'a53', 'current'),
InstrumentChannel('gpu_curr', 'gpu', 'current'),
InstrumentChannel('sys_volt', 'sys', 'voltage'),
InstrumentChannel('a57_volt', 'a57', 'voltage'),
InstrumentChannel('a53_volt', 'a53', 'voltage'),
InstrumentChannel('gpu_volt', 'gpu', 'voltage'),
InstrumentChannel('sys_pow', 'sys', 'power'),
InstrumentChannel('a57_pow', 'a57', 'power'),
InstrumentChannel('a53_pow', 'a53', 'power'),
InstrumentChannel('gpu_pow', 'gpu', 'power'),
InstrumentChannel('sys_cenr', 'sys', 'energy'),
InstrumentChannel('a57_cenr', 'a57', 'energy'),
InstrumentChannel('a53_cenr', 'a53', 'energy'),
InstrumentChannel('gpu_cenr', 'gpu', 'energy'),
]
def __init__(self, target):
super(JunoEnergyInstrument, self).__init__(target)
self.on_target_file = None
self.command = None
self.binary = self.target.bin(self.binname)
for chan in self._channels:
self.channels[chan.name] = chan
self.on_target_file = self.target.tempfile('energy', '.csv')
self.command = '{} -o {}'.format(self.binary, self.on_target_file)
def setup(self):
self.binary = self.target.install(os.path.join(PACKAGE_BIN_DIRECTORY,
self.target.abi, self.binname))
def reset(self, sites=None, kinds=None):
super(JunoEnergyInstrument, self).reset(sites, kinds)
self.target.killall(self.binname, as_root=True)
def start(self):
self.target.kick_off(self.command, as_root=True)
def stop(self):
self.target.killall(self.binname, signal='TERM', as_root=True)
def get_data(self, output_file):
temp_file = tempfile.mktemp()
self.target.pull(self.on_target_file, temp_file)
self.target.remove(self.on_target_file)
with open(temp_file, 'rb') as fh:
reader = csv.reader(fh)
headings = reader.next()
# Figure out which columns from the collected csv we actually want
select_columns = []
for chan in self.active_channels:
try:
select_columns.append(headings.index(chan.name))
except ValueError:
raise HostError('Channel "{}" is not in {}'.format(chan.name, temp_file))
with open(output_file, 'wb') as wfh:
write_headings = ['{}_{}'.format(c.site, c.kind)
for c in self.active_channels]
writer = csv.writer(wfh)
writer.writerow(write_headings)
for row in reader:
write_row = [row[c] for c in select_columns]
writer.writerow(write_row)
return MeasurementsCsv(output_file, self.active_channels)

980
devlib/target.py Normal file
View File

@@ -0,0 +1,980 @@
import os
import re
import time
import logging
import posixpath
import subprocess
import tempfile
import threading
from collections import namedtuple
from devlib.host import LocalConnection, PACKAGE_BIN_DIRECTORY
from devlib.module import get_module
from devlib.platform import Platform
from devlib.exception import TargetError, TargetNotRespondingError, TimeoutError
from devlib.utils.ssh import SshConnection
from devlib.utils.android import AdbConnection, AndroidProperties, adb_command, adb_disconnect
from devlib.utils.misc import memoized, isiterable, convert_new_lines, merge_lists
from devlib.utils.misc import ABI_MAP, get_cpu_name, ranges_to_list, escape_double_quotes
from devlib.utils.types import integer, boolean, bitmask, identifier, caseless_string
FSTAB_ENTRY_REGEX = re.compile(r'(\S+) on (\S+) type (\S+) \((\S+)\)')
ANDROID_SCREEN_STATE_REGEX = re.compile('(?:mPowerState|mScreenOn)=([0-9]+|true|false)',
re.IGNORECASE)
ANDROID_SCREEN_RESOLUTION_REGEX = re.compile(r'mUnrestrictedScreen=\(\d+,\d+\)'
r'\s+(?P<width>\d+)x(?P<height>\d+)')
DEFAULT_SHELL_PROMPT = re.compile(r'^.*(shell|root)@.*:/\S* [#$] ',
re.MULTILINE)
class Target(object):
conn_cls = None
path = None
os = None
default_modules = [
'hotplug',
'cpufreq',
'cpuidle',
'cgroups',
'hwmon',
]
@property
def core_names(self):
return self.platform.core_names
@property
def core_clusters(self):
return self.platform.core_clusters
@property
def big_core(self):
return self.platform.big_core
@property
def little_core(self):
return self.platform.little_core
@property
def is_connected(self):
return self.conn is not None
@property
@memoized
def connected_as_root(self):
result = self.execute('id')
return 'uid=0(' in result
@property
@memoized
def is_rooted(self):
if self.connected_as_root:
return True
try:
self.execute('ls /', timeout=2, as_root=True)
return True
except (TargetError, TimeoutError):
return False
@property
@memoized
def kernel_version(self):
return KernelVersion(self.execute('uname -r -v').strip())
@property
def os_version(self): # pylint: disable=no-self-use
return {}
@property
def abi(self): # pylint: disable=no-self-use
return None
@property
@memoized
def cpuinfo(self):
return Cpuinfo(self.execute('cat /proc/cpuinfo'))
@property
@memoized
def number_of_cpus(self):
num_cpus = 0
corere = re.compile(r'^\s*cpu\d+\s*$')
output = self.execute('ls /sys/devices/system/cpu')
for entry in output.split():
if corere.match(entry):
num_cpus += 1
return num_cpus
@property
@memoized
def config(self):
try:
return KernelConfig(self.execute('zcat /proc/config.gz'))
except TargetError:
for path in ['/boot/config', '/boot/config-$(uname -r)']:
try:
return KernelConfig(self.execute('cat {}'.format(path)))
except TargetError:
pass
return KernelConfig('')
@property
@memoized
def user(self):
return self.getenv('USER')
@property
def conn(self):
if self._connections:
tid = id(threading.current_thread())
if tid not in self._connections:
self._connections[tid] = self.get_connection()
return self._connections[tid]
else:
return None
def __init__(self,
connection_settings=None,
platform=None,
working_directory=None,
executables_directory=None,
connect=True,
modules=None,
load_default_modules=True,
shell_prompt=DEFAULT_SHELL_PROMPT,
):
self.connection_settings = connection_settings or {}
self.platform = platform or Platform()
self.working_directory = working_directory
self.executables_directory = executables_directory
self.modules = modules or []
self.load_default_modules = load_default_modules
self.shell_prompt = shell_prompt
self.logger = logging.getLogger(self.__class__.__name__)
self._installed_binaries = {}
self._installed_modules = {}
self._cache = {}
self._connections = {}
self.busybox = None
if load_default_modules:
module_lists = [self.default_modules]
else:
module_lists = []
module_lists += [self.modules, self.platform.modules]
self.modules = merge_lists(*module_lists, duplicates='first')
self._update_modules('early')
if connect:
self.connect()
# connection and initialization
def connect(self, timeout=None):
self.platform.init_target_connection(self)
tid = id(threading.current_thread())
self._connections[tid] = self.get_connection(timeout=timeout)
self.busybox = self.get_installed('busybox')
self._update_modules('connected')
self.platform.update_from_target(self)
if self.platform.big_core and self.load_default_modules:
self._install_module(get_module('bl'))
def disconnect(self):
for conn in self._connections.itervalues():
conn.close()
self._connections = {}
def get_connection(self, timeout=None):
if self.conn_cls is None:
raise NotImplementedError('conn_cls must be set by the subclass of Target')
return self.conn_cls(timeout=timeout, **self.connection_settings) # pylint: disable=not-callable
def setup(self, executables=None):
self.execute('mkdir -p {}'.format(self.working_directory))
self.execute('mkdir -p {}'.format(self.executables_directory))
self.busybox = self.install(os.path.join(PACKAGE_BIN_DIRECTORY, self.abi, 'busybox'))
for host_exe in (executables or []): # pylint: disable=superfluous-parens
self.install(host_exe)
def reboot(self, hard=False, connect=True, timeout=180):
if hard:
if not self.has('hard_reset'):
raise TargetError('Hard reset not supported for this target.')
self.hard_reset() # pylint: disable=no-member
else:
if not self.is_connected:
message = 'Cannot reboot target becuase it is disconnected. ' +\
'Either connect() first, or specify hard=True ' +\
'(in which case, a hard_reset module must be installed)'
raise TargetError(message)
self.reset()
if self.has('boot'):
self.boot() # pylint: disable=no-member
if connect:
self.connect(timeout=timeout)
# file transfer
def push(self, source, dest, timeout=None):
return self.conn.push(source, dest, timeout=timeout)
def pull(self, source, dest, timeout=None):
return self.conn.pull(source, dest, timeout=timeout)
# execution
def execute(self, command, timeout=None, check_exit_code=True, as_root=False):
return self.conn.execute(command, timeout, check_exit_code, as_root)
def background(self, command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, as_root=False):
return self.conn.background(command, stdout, stderr, as_root)
def invoke(self, binary, args=None, in_directory=None, on_cpus=None,
as_root=False, timeout=30):
"""
Executes the specified binary under the specified conditions.
:binary: binary to execute. Must be present and executable on the device.
:args: arguments to be passed to the binary. The can be either a list or
a string.
:in_directory: execute the binary in the specified directory. This must
be an absolute path.
:on_cpus: taskset the binary to these CPUs. This may be a single ``int`` (in which
case, it will be interpreted as the mask), a list of ``ints``, in which
case this will be interpreted as the list of cpus, or string, which
will be interpreted as a comma-separated list of cpu ranges, e.g.
``"0,4-7"``.
:as_root: Specify whether the command should be run as root
:timeout: If the invocation does not terminate within this number of seconds,
a ``TimeoutError`` exception will be raised. Set to ``None`` if the
invocation should not timeout.
"""
command = binary
if args:
if isiterable(args):
args = ' '.join(args)
command = '{} {}'.format(command, args)
if on_cpus:
on_cpus = bitmask(on_cpus)
command = '{} taskset 0x{:x} {}'.format(self.busybox, on_cpus, command)
if in_directory:
command = 'cd {} && {}'.format(in_directory, command)
return self.execute(command, as_root=as_root, timeout=timeout)
def kick_off(self, command, as_root=False):
raise NotImplementedError()
# sysfs interaction
def read_value(self, path, kind=None):
output = self.execute('cat \'{}\''.format(path), as_root=self.is_rooted).strip() # pylint: disable=E1103
if kind:
return kind(output)
else:
return output
def read_int(self, path):
return self.read_value(path, kind=integer)
def read_bool(self, path):
return self.read_value(path, kind=boolean)
def write_value(self, path, value, verify=True):
value = str(value)
self.execute('echo {} > \'{}\''.format(value, path), check_exit_code=False, as_root=True)
if verify:
output = self.read_value(path)
if not output == value:
message = 'Could not set the value of {} to "{}" (read "{}")'.format(path, value, output)
raise TargetError(message)
def reset(self):
try:
self.execute('reboot', as_root=self.is_rooted, timeout=2)
except (TargetError, TimeoutError, subprocess.CalledProcessError):
# on some targets "reboot" doesn't return gracefully
pass
def check_responsive(self):
try:
self.conn.execute('ls /', timeout=5)
except (TimeoutError, subprocess.CalledProcessError):
raise TargetNotRespondingError(self.conn.name)
# process management
def kill(self, pid, signal=None, as_root=False):
signal_string = '-s {}'.format(signal) if signal else ''
self.execute('kill {} {}'.format(signal_string, pid), as_root=as_root)
def killall(self, process_name, signal=None, as_root=False):
for pid in self.get_pids_of(process_name):
self.kill(pid, signal=signal, as_root=as_root)
def get_pids_of(self, process_name):
raise NotImplementedError()
def ps(self, **kwargs):
raise NotImplementedError()
# files
def file_exists(self, filepath):
command = 'if [ -e \'{}\' ]; then echo 1; else echo 0; fi'
return boolean(self.execute(command.format(filepath)).strip())
def list_file_systems(self):
output = self.execute('mount')
fstab = []
for line in output.split('\n'):
line = line.strip()
if not line:
continue
match = FSTAB_ENTRY_REGEX.search(line)
if match:
fstab.append(FstabEntry(match.group(1), match.group(2),
match.group(3), match.group(4),
None, None))
else: # assume pre-M Android
fstab.append(FstabEntry(*line.split()))
return fstab
def list_directory(self, path, as_root=False):
raise NotImplementedError()
def get_workpath(self, name):
return self.path.join(self.working_directory, name)
def tempfile(self, prefix='', suffix=''):
names = tempfile._get_candidate_names() # pylint: disable=W0212
for _ in xrange(tempfile.TMP_MAX):
name = names.next()
path = self.get_workpath(prefix + name + suffix)
if not self.file_exists(path):
return path
raise IOError('No usable temporary filename found')
def remove(self, path, as_root=False):
self.execute('rm -rf {}'.format(path), as_root=as_root)
# misc
def core_cpus(self, core):
return [i for i, c in enumerate(self.core_names) if c == core]
def list_online_cpus(self, core=None):
path = self.path.join('/sys/devices/system/cpu/online')
output = self.read_value(path)
all_online = ranges_to_list(output)
if core:
cpus = self.core_cpus(core)
if not cpus:
raise ValueError(core)
return [o for o in all_online if o in cpus]
else:
return all_online
def list_offline_cpus(self):
online = self.list_online_cpus()
return [c for c in xrange(self.number_of_cpus)
if c not in online]
def getenv(self, variable):
return self.execute('echo ${}'.format(variable)).rstrip('\r\n')
def capture_screen(self, filepath):
raise NotImplementedError()
def install(self, filepath, timeout=None, with_name=None):
raise NotImplementedError()
def uninstall(self, name):
raise NotImplementedError()
def get_installed(self, name):
for path in self.getenv('PATH').split(self.path.pathsep):
try:
if name in self.list_directory(path):
return self.path.join(path, name)
except TargetError:
pass # directory does not exist or no executable premssions
if name in self.list_directory(self.executables_directory):
return self.path.join(self.executables_directory, name)
which = get_installed
def is_installed(self, name):
return bool(self.get_installed(name))
def bin(self, name):
return self._installed_binaries.get(name, name)
def has(self, modname):
return hasattr(self, identifier(modname))
def _update_modules(self, stage):
for mod in self.modules:
if isinstance(mod, dict):
mod, params = mod.items()[0]
else:
params = {}
mod = get_module(mod)
if not mod.stage == stage:
continue
if mod.probe(self):
self._install_module(mod, **params)
else:
self.logger.debug('Module {} is not supported by the target'.format(mod.name))
def _install_module(self, mod, **params):
if mod.name not in self._installed_modules:
self.logger.debug('Installing module {}'.format(mod.name))
mod.install(self, **params)
self._installed_modules[mod.name] = mod
else:
self.logger.debug('Module {} is already installed.'.format(mod.name))
class LinuxTarget(Target):
conn_cls = SshConnection
path = posixpath
os = 'linux'
@property
@memoized
def abi(self):
value = self.execute('uname -m').strip()
for abi, architectures in ABI_MAP.iteritems():
if value in architectures:
result = abi
break
else:
result = value
return result
@property
@memoized
def os_version(self):
os_version = {}
try:
command = 'ls /etc/*-release /etc*-version /etc/*_release /etc/*_version 2>/dev/null'
version_files = self.execute(command, check_exit_code=False).strip().split()
for vf in version_files:
name = self.path.basename(vf)
output = self.read_value(vf)
os_version[name] = output.strip().replace('\n', ' ')
except TargetError:
raise
return os_version
def connect(self, timeout=None):
super(LinuxTarget, self).connect(timeout=timeout)
if self.working_directory is None:
if self.connected_as_root:
self.working_directory = '/root/devlib-target'
else:
self.working_directory = '/home/{}/devlib-target'.format(self.user)
if self.executables_directory is None:
self.executables_directory = self.path.join(self.working_directory, 'bin')
def kick_off(self, command, as_root=False):
command = 'sh -c "{}" 1>/dev/null 2>/dev/null &'.format(escape_double_quotes(command))
return self.conn.execute(command, as_root=as_root)
def get_pids_of(self, process_name):
"""Returns a list of PIDs of all processes with the specified name."""
# result should be a column of PIDs with the first row as "PID" header
result = self.execute('ps -C {} -o pid'.format(process_name), # NOQA
check_exit_code=False).strip().split()
if len(result) >= 2: # at least one row besides the header
return map(int, result[1:])
else:
return []
def ps(self, **kwargs):
command = 'ps -eo user,pid,ppid,vsize,rss,wchan,pcpu,state,fname'
lines = iter(convert_new_lines(self.execute(command)).split('\n'))
lines.next() # header
result = []
for line in lines:
parts = re.split(r'\s+', line, maxsplit=8)
if parts and parts != ['']:
result.append(PsEntry(*(parts[0:1] + map(int, parts[1:5]) + parts[5:])))
if not kwargs:
return result
else:
filtered_result = []
for entry in result:
if all(getattr(entry, k) == v for k, v in kwargs.iteritems()):
filtered_result.append(entry)
return filtered_result
def list_directory(self, path, as_root=False):
contents = self.execute('ls -1 {}'.format(path), as_root=as_root)
return [x.strip() for x in contents.split('\n') if x.strip()]
def install(self, filepath, timeout=None, with_name=None): # pylint: disable=W0221
destpath = self.path.join(self.executables_directory,
with_name and with_name or self.path.basename(filepath))
self.push(filepath, destpath)
self.execute('chmod a+x {}'.format(destpath), timeout=timeout)
self._installed_binaries[self.path.basename(destpath)] = destpath
return destpath
def uninstall(self, name):
path = self.path.join(self.executables_directory, name)
self.remove(path)
def capture_screen(self, filepath):
if not self.is_installed('scrot'):
self.logger.debug('Could not take screenshot as scrot is not installed.')
return
try:
tmpfile = self.tempfile()
self.execute('DISPLAY=:0.0 scrot {}'.format(tmpfile))
self.pull(tmpfile, filepath)
self.remove(tmpfile)
except TargetError as e:
if "Can't open X dispay." not in e.message:
raise e
message = e.message.split('OUTPUT:', 1)[1].strip() # pylint: disable=no-member
self.logger.debug('Could not take screenshot: {}'.format(message))
class AndroidTarget(Target):
conn_cls = AdbConnection
path = posixpath
os = 'android'
@property
@memoized
def abi(self):
return self.getprop()['ro.product.cpu.abi'].split('-')[0]
@property
@memoized
def os_version(self):
os_version = {}
for k, v in self.getprop().iteritems():
if k.startswith('ro.build.version'):
part = k.split('.')[-1]
os_version[part] = v
return os_version
@property
def adb_name(self):
return self.conn.device
@property
@memoized
def screen_resolution(self):
output = self.execute('dumpsys window')
match = ANDROID_SCREEN_RESOLUTION_REGEX.search(output)
if match:
return (int(match.group('width')),
int(match.group('height')))
else:
return (0, 0)
def __init__(self, *args, **kwargs):
super(AndroidTarget, self).__init__(*args, **kwargs)
self._file_transfer_cache = None
def reset(self, fastboot=False): # pylint: disable=arguments-differ
try:
self.execute('reboot {}'.format(fastboot and 'fastboot' or ''),
as_root=self.is_rooted, timeout=2)
except (TargetError, TimeoutError, subprocess.CalledProcessError):
# on some targets "reboot" doesn't return gracefully
pass
def connect(self, timeout=10, check_boot_completed=True): # pylint: disable=arguments-differ
start = time.time()
device = self.connection_settings.get('device')
if device and ':' in device:
# ADB does not automatically remove a network device from it's
# devices list when the connection is broken by the remote, so the
# adb connection may have gone "stale", resulting in adb blocking
# indefinitely when making calls to the device. To avoid this,
# always disconnect first.
adb_disconnect(device)
super(AndroidTarget, self).connect(timeout=timeout)
if self.working_directory is None:
self.working_directory = '/data/local/tmp/devlib-target'
self._file_transfer_cache = self.path.join(self.working_directory, '.file-cache')
if self.executables_directory is None:
self.executables_directory = self.path.join(self.working_directory, 'bin')
if check_boot_completed:
boot_completed = boolean(self.getprop('sys.boot_completed'))
while not boot_completed and timeout >= time.time() - start:
time.sleep(5)
boot_completed = boolean(self.getprop('sys.boot_completed'))
if not boot_completed:
raise TargetError('Connected but Android did not fully boot.')
def setup(self, executables=None):
super(AndroidTarget, self).setup(executables)
self.execute('mkdir -p {}'.format(self._file_transfer_cache))
def kick_off(self, command, as_root=False):
"""
Like execute but closes adb session and returns immediately, leaving the command running on the
device (this is different from execute(background=True) which keeps adb connection open and returns
a subprocess object).
.. note:: This relies on busybox's nohup applet and so won't work on unrooted devices.
"""
if not self.is_rooted:
raise TargetError('kick_off uses busybox\'s nohup applet and so can only be run a rooted device.')
try:
command = 'cd {} && {} nohup {}'.format(self.working_directory, self.bin('busybox'), command)
output = self.execute(command, timeout=1, as_root=as_root)
except TimeoutError:
pass
else:
raise ValueError('Background command exited before timeout; got "{}"'.format(output))
def list_directory(self, path, as_root=False):
contents = self.execute('ls {}'.format(path), as_root=as_root)
return [x.strip() for x in contents.split('\n') if x.strip()]
def install(self, filepath, timeout=None, with_name=None): # pylint: disable=W0221
ext = os.path.splitext(filepath)[1].lower()
if ext == '.apk':
return self.install_apk(filepath, timeout)
else:
return self.install_executable(filepath, with_name)
def uninstall(self, name):
if self.package_is_installed(name):
self.uninstall_package(name)
else:
self.uninstall_executable(name)
def get_pids_of(self, process_name):
result = self.execute('ps {}'.format(process_name[-15:]), check_exit_code=False).strip()
if result and 'not found' not in result:
return [int(x.split()[1]) for x in result.split('\n')[1:]]
else:
return []
def ps(self, **kwargs):
lines = iter(convert_new_lines(self.execute('ps')).split('\n'))
lines.next() # header
result = []
for line in lines:
parts = line.split()
if parts:
result.append(PsEntry(*(parts[0:1] + map(int, parts[1:5]) + parts[5:])))
if not kwargs:
return result
else:
filtered_result = []
for entry in result:
if all(getattr(entry, k) == v for k, v in kwargs.iteritems()):
filtered_result.append(entry)
return filtered_result
def capture_screen(self, filepath):
on_device_file = self.path.join(self.working_directory, 'screen_capture.png')
self.execute('screencap -p {}'.format(on_device_file))
self.pull(on_device_file, filepath)
self.remove(on_device_file)
def push(self, source, dest, as_root=False, timeout=None): # pylint: disable=arguments-differ
if not as_root:
self.conn.push(source, dest, timeout=timeout)
else:
device_tempfile = self.path.join(self._file_transfer_cache, source.lstrip(self.path.sep))
self.execute('mkdir -p {}'.format(self.path.dirname(device_tempfile)))
self.conn.push(source, device_tempfile, timeout=timeout)
self.execute('cp {} {}'.format(device_tempfile, dest), as_root=True)
def pull(self, source, dest, as_root=False, timeout=None): # pylint: disable=arguments-differ
if not as_root:
self.conn.pull(source, dest, timeout=timeout)
else:
device_tempfile = self.path.join(self._file_transfer_cache, source.lstrip(self.path.sep))
self.execute('mkdir -p {}'.format(self.path.dirname(device_tempfile)))
self.execute('cp {} {}'.format(source, device_tempfile), as_root=True)
self.conn.pull(device_tempfile, dest, timeout=timeout)
# Android-specific
def swipe_to_unlock(self):
width, height = self.screen_resolution
swipe_heigh = height * 2 // 3
start = 100
stop = width - start
command = 'input swipe {} {} {} {}'
self.execute(command.format(start, swipe_heigh, stop, swipe_heigh))
def getprop(self, prop=None):
props = AndroidProperties(self.execute('getprop'))
if prop:
return props[prop]
return props
def is_installed(self, name):
return super(AndroidTarget, self).is_installed(name) or self.package_is_installed(name)
def package_is_installed(self, package_name):
return package_name in self.list_packages()
def list_packages(self):
output = self.execute('pm list packages')
output = output.replace('package:', '')
return output.split()
def get_package_version(self, package):
output = self.execute('dumpsys package {}'.format(package))
for line in convert_new_lines(output).split('\n'):
if 'versionName' in line:
return line.split('=', 1)[1]
return None
def install_apk(self, filepath, timeout=None): # pylint: disable=W0221
ext = os.path.splitext(filepath)[1].lower()
if ext == '.apk':
return adb_command(self.adb_name, "install {}".format(filepath), timeout=timeout)
else:
raise TargetError('Can\'t install {}: unsupported format.'.format(filepath))
def install_executable(self, filepath, with_name=None):
self._ensure_executables_directory_is_writable()
executable_name = with_name or os.path.basename(filepath)
on_device_file = self.path.join(self.working_directory, executable_name)
on_device_executable = self.path.join(self.executables_directory, executable_name)
self.push(filepath, on_device_file)
if on_device_file != on_device_executable:
self.execute('cp {} {}'.format(on_device_file, on_device_executable), as_root=self.is_rooted)
self.remove(on_device_file, as_root=self.is_rooted)
self.execute('chmod 0777 {}'.format(on_device_executable), as_root=self.is_rooted)
self._installed_binaries[executable_name] = on_device_executable
return on_device_executable
def uninstall_package(self, package):
adb_command(self.adb_name, "uninstall {}".format(package), timeout=30)
def uninstall_executable(self, executable_name):
on_device_executable = self.path.join(self.executables_directory, executable_name)
self._ensure_executables_directory_is_writable()
self.remove(on_device_executable, as_root=self.is_rooted)
def dump_logcat(self, filepath, filter=None, append=False, timeout=30): # pylint: disable=redefined-builtin
op = '>>' if append == True else '>'
filtstr = ' -s {}'.format(filter) if filter else ''
command = 'logcat -d{} {} {}'.format(filtstr, op, filepath)
adb_command(self.adb_name, command, timeout=timeout)
def clear_logcat(self):
adb_command(self.adb_name, 'logcat -c', timeout=30)
def is_screen_on(self):
output = self.execute('dumpsys power')
match = ANDROID_SCREEN_STATE_REGEX.search(output)
if match:
return boolean(match.group(1))
else:
raise TargetError('Could not establish screen state.')
def ensure_screen_is_on(self):
if not self.is_screen_on():
self.execute('input keyevent 26')
def _ensure_executables_directory_is_writable(self):
matched = []
for entry in self.list_file_systems():
if self.executables_directory.rstrip('/').startswith(entry.mount_point):
matched.append(entry)
if matched:
entry = sorted(matched, key=lambda x: len(x.mount_point))[-1]
if 'rw' not in entry.options:
self.execute('mount -o rw,remount {} {}'.format(entry.device,
entry.mount_point),
as_root=True)
else:
message = 'Could not find mount point for executables directory {}'
raise TargetError(message.format(self.executables_directory))
FstabEntry = namedtuple('FstabEntry', ['device', 'mount_point', 'fs_type', 'options', 'dump_freq', 'pass_num'])
PsEntry = namedtuple('PsEntry', 'user pid ppid vsize rss wchan pc state name')
class Cpuinfo(object):
@property
@memoized
def architecture(self):
for section in self.sections:
if 'CPU architecture' in section:
return section['CPU architecture']
if 'architecture' in section:
return section['architecture']
@property
@memoized
def cpu_names(self):
cpu_names = []
global_name = None
for section in self.sections:
if 'processor' in section:
if 'CPU part' in section:
cpu_names.append(_get_part_name(section))
elif 'model name' in section:
cpu_names.append(_get_model_name(section))
else:
cpu_names.append(None)
elif 'CPU part' in section:
global_name = _get_part_name(section)
return [caseless_string(c or global_name) for c in cpu_names]
def __init__(self, text):
self.sections = None
self.text = None
self.parse(text)
@memoized
def get_cpu_features(self, cpuid=0):
global_features = []
for section in self.sections:
if 'processor' in section:
if int(section.get('processor')) != cpuid:
continue
if 'Features' in section:
return section.get('Features').split()
elif 'Features' in section:
global_features = section.get('Features').split()
return global_features
def parse(self, text):
self.sections = []
current_section = {}
self.text = text.strip()
for line in self.text.split('\n'):
line = line.strip()
if line:
key, value = line.split(':', 1)
current_section[key.strip()] = value.strip()
else: # not line
self.sections.append(current_section)
current_section = {}
self.sections.append(current_section)
def __str__(self):
return 'CpuInfo({})'.format(self.cpu_names)
__repr__ = __str__
class KernelVersion(object):
def __init__(self, version_string):
if ' #' in version_string:
release, version = version_string.split(' #')
self.release = release
self.version = version
elif version_string.startswith('#'):
self.release = ''
self.version = version_string
else:
self.release = version_string
self.version = ''
def __str__(self):
return '{} {}'.format(self.release, self.version)
__repr__ = __str__
class KernelConfig(object):
not_set_regex = re.compile(r'# (\S+) is not set')
@staticmethod
def get_config_name(name):
name = name.upper()
if not name.startswith('CONFIG_'):
name = 'CONFIG_' + name
return name
def iteritems(self):
return self._config.iteritems()
def __init__(self, text):
self.text = text
self._config = {}
for line in text.split('\n'):
line = line.strip()
if line.startswith('#'):
match = self.not_set_regex.search(line)
if match:
self._config[match.group(1)] = 'n'
elif '=' in line:
name, value = line.split('=', 1)
self._config[name.strip()] = value.strip()
def get(self, name):
return self._config.get(self.get_config_name(name))
def like(self, name):
regex = re.compile(name, re.I)
result = {}
for k, v in self._config.iteritems():
if regex.search(k):
result[k] = v
return result
def is_enabled(self, name):
return self.get(name) == 'y'
def is_module(self, name):
return self.get(name) == 'm'
def is_not_set(self, name):
return self.get(name) == 'n'
def has(self, name):
return self.get(name) in ['m', 'y']
class LocalLinuxTarget(LinuxTarget):
conn_cls = LocalConnection
def connect(self, timeout=None):
if self.working_directory is None:
self.working_directory = '/tmp'
if self.executables_directory is None:
self.executables_directory = '/tmp'
super(LocalLinuxTarget, self).connect(timeout)
def _get_model_name(section):
name_string = section['model name']
parts = name_string.split('@')[0].strip().split()
return ' '.join([p for p in parts
if '(' not in p and p != 'CPU'])
def _get_part_name(section):
implementer = section.get('CPU implementer', '0x0')
part = section['CPU part']
variant = section.get('CPU variant', '0x0')
name = get_cpu_name(*map(integer, [implementer, part, variant]))
if name is None:
name = '{}/{}/{}'.format(implementer, part, variant)
return name

20
devlib/trace/__init__.py Normal file
View File

@@ -0,0 +1,20 @@
import logging
class TraceCollector(object):
def __init__(self, target):
self.target = target
self.logger = logging.getLogger(self.__class__.__name__)
def reset(self):
pass
def start(self):
pass
def stop(self):
pass
def get_trace(self, outfile):
pass

199
devlib/trace/ftrace.py Normal file
View File

@@ -0,0 +1,199 @@
# Copyright 2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import division
import os
import time
import subprocess
from devlib.trace import TraceCollector
from devlib.host import PACKAGE_BIN_DIRECTORY
from devlib.exception import TargetError, HostError
from devlib.utils.misc import check_output, which
TRACE_MARKER_START = 'TRACE_MARKER_START'
TRACE_MARKER_STOP = 'TRACE_MARKER_STOP'
OUTPUT_TRACE_FILE = 'trace.dat'
DEFAULT_EVENTS = [
'cpu_frequency',
'cpu_idle',
'sched_migrate_task',
'sched_process_exec',
'sched_process_fork',
'sched_stat_iowait',
'sched_switch',
'sched_wakeup',
'sched_wakeup_new',
]
TIMEOUT = 180
class FtraceCollector(TraceCollector):
def __init__(self, target,
events=None,
buffer_size=None,
buffer_size_step=1000,
buffer_size_file='/sys/kernel/debug/tracing/buffer_size_kb',
marker_file='/sys/kernel/debug/tracing/trace_marker',
automark=True,
autoreport=True,
autoview=False,
no_install=False,
):
super(FtraceCollector, self).__init__(target)
self.events = events if events is not None else DEFAULT_EVENTS
self.buffer_size = buffer_size
self.buffer_size_step = buffer_size_step
self.buffer_size_file = buffer_size_file
self.marker_file = marker_file
self.automark = automark
self.autoreport = autoreport
self.autoview = autoview
self.target_output_file = os.path.join(self.target.working_directory, OUTPUT_TRACE_FILE)
self.target_binary = None
self.host_binary = None
self.start_time = None
self.stop_time = None
self.event_string = _build_trace_events(self.events)
self._reset_needed = True
self.host_binary = which('trace-cmd')
self.kernelshark = which('kernelshark')
if not self.target.is_rooted:
raise TargetError('trace-cmd instrument cannot be used on an unrooted device.')
if self.autoreport and self.host_binary is None:
raise HostError('trace-cmd binary must be installed on the host if autoreport=True.')
if self.autoview and self.kernelshark is None:
raise HostError('kernelshark binary must be installed on the host if autoview=True.')
if not no_install:
host_file = os.path.join(PACKAGE_BIN_DIRECTORY, self.target.abi, 'trace-cmd')
self.target_binary = self.target.install(host_file)
else:
if not self.target.is_installed('trace-cmd'):
raise TargetError('No trace-cmd found on device and no_install=True is specified.')
self.target_binary = 'trace-cmd'
def reset(self):
if self.buffer_size:
self._set_buffer_size()
self.target.execute('{} reset'.format(self.target_binary), as_root=True, timeout=TIMEOUT)
self._reset_needed = False
def start(self):
self.start_time = time.time()
if self._reset_needed:
self.reset()
if self.automark:
self.mark_start()
self.target.execute('{} start {}'.format(self.target_binary, self.event_string), as_root=True)
def stop(self):
self.stop_time = time.time()
if self.automark:
self.mark_stop()
self.target.execute('{} stop'.format(self.target_binary), timeout=TIMEOUT, as_root=True)
self._reset_needed = True
def get_trace(self, outfile):
if os.path.isdir(outfile):
outfile = os.path.join(outfile, os.path.dirname(self.target_output_file))
self.target.execute('{} extract -o {}'.format(self.target_binary, self.target_output_file),
timeout=TIMEOUT, as_root=True)
# The size of trace.dat will depend on how long trace-cmd was running.
# Therefore timout for the pull command must also be adjusted
# accordingly.
pull_timeout = self.stop_time - self.start_time
self.target.pull(self.target_output_file, outfile, timeout=pull_timeout)
if not os.path.isfile(outfile):
self.logger.warning('Binary trace not pulled from device.')
else:
if self.autoreport:
textfile = os.path.splitext(outfile)[0] + '.txt'
self.report(outfile, textfile)
if self.autoview:
self.view(outfile)
def report(self, binfile, destfile):
# To get the output of trace.dat, trace-cmd must be installed
# This is done host-side because the generated file is very large
try:
command = '{} report {} > {}'.format(self.host_binary, binfile, destfile)
self.logger.debug(command)
process = subprocess.Popen(command, stderr=subprocess.PIPE, shell=True)
_, error = process.communicate()
if process.returncode:
raise TargetError('trace-cmd returned non-zero exit code {}'.format(process.returncode))
if error:
# logged at debug level, as trace-cmd always outputs some
# errors that seem benign.
self.logger.debug(error)
if os.path.isfile(destfile):
self.logger.debug('Verifying traces.')
with open(destfile) as fh:
for line in fh:
if 'EVENTS DROPPED' in line:
self.logger.warning('Dropped events detected.')
break
else:
self.logger.debug('Trace verified.')
else:
self.logger.warning('Could not generate trace.txt.')
except OSError:
raise HostError('Could not find trace-cmd. Please make sure it is installed and is in PATH.')
def view(self, binfile):
check_output('{} {}'.format(self.kernelshark, binfile), shell=True)
def teardown(self):
self.target.remove(self.target.path.join(self.target.working_directory, OUTPUT_TRACE_FILE))
def mark_start(self):
self.target.write_value(self.marker_file, TRACE_MARKER_START, verify=False)
def mark_stop(self):
self.target.write_value(self.marker_file, TRACE_MARKER_STOP, verify=False)
def _set_buffer_size(self):
target_buffer_size = self.buffer_size
attempt_buffer_size = target_buffer_size
buffer_size = 0
floor = 1000 if target_buffer_size > 1000 else target_buffer_size
while attempt_buffer_size >= floor:
self.target.write_value(self.buffer_size_file, attempt_buffer_size, verify=False)
buffer_size = self.target.read_int(self.buffer_size_file)
if buffer_size == attempt_buffer_size:
break
else:
attempt_buffer_size -= self.buffer_size_step
if buffer_size == target_buffer_size:
return
while attempt_buffer_size < target_buffer_size:
attempt_buffer_size += self.buffer_size_step
self.target.write_value(self.buffer_size_file, attempt_buffer_size, verify=False)
buffer_size = self.target.read_int(self.buffer_size_file)
if attempt_buffer_size != buffer_size:
message = 'Failed to set trace buffer size to {}, value set was {}'
self.logger.warning(message.format(target_buffer_size, buffer_size))
break
def _build_trace_events(events):
event_string = ' '.join(['-e {}'.format(e) for e in events])
return event_string

16
devlib/utils/__init__.py Normal file
View File

@@ -0,0 +1,16 @@
# Copyright 2013-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

428
devlib/utils/android.py Normal file
View File

@@ -0,0 +1,428 @@
# Copyright 2013-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Utility functions for working with Android devices through adb.
"""
# pylint: disable=E1103
import os
import time
import subprocess
import logging
import re
from collections import defaultdict
from devlib.exception import TargetError, HostError
from devlib.utils.misc import check_output, which
from devlib.utils.misc import escape_single_quotes, escape_double_quotes
logger = logging.getLogger('android')
MAX_ATTEMPTS = 5
AM_START_ERROR = re.compile(r"Error: Activity class {[\w|.|/]*} does not exist")
# See:
# http://developer.android.com/guide/topics/manifest/uses-sdk-element.html#ApiLevels
ANDROID_VERSION_MAP = {
22: 'LOLLYPOP_MR1',
21: 'LOLLYPOP',
20: 'KITKAT_WATCH',
19: 'KITKAT',
18: 'JELLY_BEAN_MR2',
17: 'JELLY_BEAN_MR1',
16: 'JELLY_BEAN',
15: 'ICE_CREAM_SANDWICH_MR1',
14: 'ICE_CREAM_SANDWICH',
13: 'HONEYCOMB_MR2',
12: 'HONEYCOMB_MR1',
11: 'HONEYCOMB',
10: 'GINGERBREAD_MR1',
9: 'GINGERBREAD',
8: 'FROYO',
7: 'ECLAIR_MR1',
6: 'ECLAIR_0_1',
5: 'ECLAIR',
4: 'DONUT',
3: 'CUPCAKE',
2: 'BASE_1_1',
1: 'BASE',
}
# Initialized in functions near the botton of the file
android_home = None
platform_tools = None
adb = None
aapt = None
fastboot = None
class AndroidProperties(object):
def __init__(self, text):
self._properties = {}
self.parse(text)
def parse(self, text):
self._properties = dict(re.findall(r'\[(.*?)\]:\s+\[(.*?)\]', text))
def iteritems(self):
return self._properties.iteritems()
def __iter__(self):
return iter(self._properties)
def __getattr__(self, name):
return self._properties.get(name)
__getitem__ = __getattr__
class AdbDevice(object):
def __init__(self, name, status):
self.name = name
self.status = status
def __cmp__(self, other):
if isinstance(other, AdbDevice):
return cmp(self.name, other.name)
else:
return cmp(self.name, other)
def __str__(self):
return 'AdbDevice({}, {})'.format(self.name, self.status)
__repr__ = __str__
class ApkInfo(object):
version_regex = re.compile(r"name='(?P<name>[^']+)' versionCode='(?P<vcode>[^']+)' versionName='(?P<vname>[^']+)'")
name_regex = re.compile(r"name='(?P<name>[^']+)'")
def __init__(self, path=None):
self.path = path
self.package = None
self.activity = None
self.label = None
self.version_name = None
self.version_code = None
self.parse(path)
def parse(self, apk_path):
_check_env()
command = [aapt, 'dump', 'badging', apk_path]
logger.debug(' '.join(command))
output = subprocess.check_output(command)
for line in output.split('\n'):
if line.startswith('application-label:'):
self.label = line.split(':')[1].strip().replace('\'', '')
elif line.startswith('package:'):
match = self.version_regex.search(line)
if match:
self.package = match.group('name')
self.version_code = match.group('vcode')
self.version_name = match.group('vname')
elif line.startswith('launchable-activity:'):
match = self.name_regex.search(line)
self.activity = match.group('name')
else:
pass # not interested
class AdbConnection(object):
# maintains the count of parallel active connections to a device, so that
# adb disconnect is not invoked untill all connections are closed
active_connections = defaultdict(int)
@property
def name(self):
return self.device
def __init__(self, device=None, timeout=10):
self.timeout = timeout
if device is None:
device = adb_get_device(timeout=timeout)
self.device = device
adb_connect(self.device)
AdbConnection.active_connections[self.device] += 1
def push(self, source, dest, timeout=None):
if timeout is None:
timeout = self.timeout
command = 'push {} {}'.format(source, dest)
return adb_command(self.device, command, timeout=timeout)
def pull(self, source, dest, timeout=None):
if timeout is None:
timeout = self.timeout
command = 'pull {} {}'.format(source, dest)
return adb_command(self.device, command, timeout=timeout)
def execute(self, command, timeout=None, check_exit_code=False, as_root=False):
return adb_shell(self.device, command, timeout, check_exit_code, as_root)
def background(self, command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, as_root=False):
return adb_background_shell(self.device, command, stdout, stderr, as_root)
def close(self):
AdbConnection.active_connections[self.device] -= 1
if AdbConnection.active_connections[self.device] <= 0:
adb_disconnect(self.device)
del AdbConnection.active_connections[self.device]
def cancel_running_command(self):
# adbd multiplexes commands so that they don't interfer with each
# other, so there is no need to explicitly cancel a running command
# before the next one can be issued.
pass
def fastboot_command(command, timeout=None):
_check_env()
full_command = "fastboot {}".format(command)
logger.debug(full_command)
output, _ = check_output(full_command, timeout, shell=True)
return output
def fastboot_flash_partition(partition, path_to_image):
command = 'flash {} {}'.format(partition, path_to_image)
fastboot_command(command)
def adb_get_device(timeout=None):
"""
Returns the serial number of a connected android device.
If there are more than one device connected to the machine, or it could not
find any device connected, :class:`devlib.exceptions.HostError` is raised.
"""
# TODO this is a hacky way to issue a adb command to all listed devices
# The output of calling adb devices consists of a heading line then
# a list of the devices sperated by new line
# The last line is a blank new line. in otherwords, if there is a device found
# then the output length is 2 + (1 for each device)
start = time.time()
while True:
output = adb_command(None, "devices").splitlines() # pylint: disable=E1103
output_length = len(output)
if output_length == 3:
# output[1] is the 2nd line in the output which has the device name
# Splitting the line by '\t' gives a list of two indexes, which has
# device serial in 0 number and device type in 1.
return output[1].split('\t')[0]
elif output_length > 3:
message = '{} Android devices found; either explicitly specify ' +\
'the device you want, or make sure only one is connected.'
raise HostError(message.format(output_length - 2))
else:
if timeout < time.time() - start:
raise HostError('No device is connected and available')
time.sleep(1)
def adb_connect(device, timeout=None, attempts=MAX_ATTEMPTS):
_check_env()
tries = 0
output = None
while tries <= attempts:
tries += 1
if device:
command = 'adb connect {}'.format(device)
logger.debug(command)
output, _ = check_output(command, shell=True, timeout=timeout)
if _ping(device):
break
time.sleep(10)
else: # did not connect to the device
message = 'Could not connect to {}'.format(device or 'a device')
if output:
message += '; got: "{}"'.format(output)
raise HostError(message)
def adb_disconnect(device):
_check_env()
if not device:
return
if ":" in device:
command = "adb disconnect " + device
logger.debug(command)
retval = subprocess.call(command, stdout=open(os.devnull, 'wb'), shell=True)
if retval:
raise TargetError('"{}" returned {}'.format(command, retval))
def _ping(device):
_check_env()
device_string = ' -s {}'.format(device) if device else ''
command = "adb{} shell \"ls / > /dev/null\"".format(device_string)
logger.debug(command)
result = subprocess.call(command, stderr=subprocess.PIPE, shell=True)
if not result:
return True
else:
return False
def adb_shell(device, command, timeout=None, check_exit_code=False, as_root=False): # NOQA
_check_env()
if as_root:
command = 'echo "{}" | su'.format(escape_double_quotes(command))
device_string = ' -s {}'.format(device) if device else ''
full_command = 'adb{} shell "{}"'.format(device_string,
escape_double_quotes(command))
logger.debug(full_command)
if check_exit_code:
actual_command = "adb{} shell '({}); echo $?'".format(device_string,
escape_single_quotes(command))
raw_output, error = check_output(actual_command, timeout, shell=True)
if raw_output:
try:
output, exit_code, _ = raw_output.rsplit('\n', 2)
except ValueError:
exit_code, _ = raw_output.rsplit('\n', 1)
output = ''
else: # raw_output is empty
exit_code = '969696' # just because
output = ''
exit_code = exit_code.strip()
if exit_code.isdigit():
if int(exit_code):
message = 'Got exit code {}\nfrom: {}\nSTDOUT: {}\nSTDERR: {}'
raise TargetError(message.format(exit_code, full_command, output, error))
elif AM_START_ERROR.findall(output):
message = 'Could not start activity; got the following:'
message += '\n{}'.format(AM_START_ERROR.findall(output)[0])
raise TargetError(message)
else: # not all digits
if AM_START_ERROR.findall(output):
message = 'Could not start activity; got the following:\n{}'
raise TargetError(message.format(AM_START_ERROR.findall(output)[0]))
else:
message = 'adb has returned early; did not get an exit code. '\
'Was kill-server invoked?'
raise TargetError(message)
else: # do not check exit code
output, _ = check_output(full_command, timeout, shell=True)
return output
def adb_background_shell(device, command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
as_root=False):
"""Runs the sepcified command in a subprocess, returning the the Popen object."""
_check_env()
if as_root:
command = 'echo \'{}\' | su'.format(escape_single_quotes(command))
device_string = ' -s {}'.format(device) if device else ''
full_command = 'adb{} shell "{}"'.format(device_string, escape_double_quotes(command))
logger.debug(full_command)
return subprocess.Popen(full_command, stdout=stdout, stderr=stderr, shell=True)
def adb_list_devices():
output = adb_command(None, 'devices')
devices = []
for line in output.splitlines():
parts = [p.strip() for p in line.split()]
if len(parts) == 2:
devices.append(AdbDevice(*parts))
return devices
def adb_command(device, command, timeout=None):
_check_env()
device_string = ' -s {}'.format(device) if device else ''
full_command = "adb{} {}".format(device_string, command)
logger.debug(full_command)
output, _ = check_output(full_command, timeout, shell=True)
return output
# Messy environment initialisation stuff...
class _AndroidEnvironment(object):
def __init__(self):
self.android_home = None
self.platform_tools = None
self.adb = None
self.aapt = None
self.fastboot = None
def _initialize_with_android_home(env):
logger.debug('Using ANDROID_HOME from the environment.')
env.android_home = android_home
env.platform_tools = os.path.join(android_home, 'platform-tools')
os.environ['PATH'] += os.pathsep + env.platform_tools
_init_common(env)
return env
def _initialize_without_android_home(env):
if which('adb'):
env.adb = 'adb'
else:
raise HostError('ANDROID_HOME is not set and adb is not in PATH. '
'Have you installed Android SDK?')
logger.debug('Discovering ANDROID_HOME from adb path.')
env.platform_tools = os.path.dirname(env.adb)
env.android_home = os.path.dirname(env.platform_tools)
_init_common(env)
return env
def _init_common(env):
logger.debug('ANDROID_HOME: {}'.format(env.android_home))
build_tools_directory = os.path.join(env.android_home, 'build-tools')
if not os.path.isdir(build_tools_directory):
msg = '''ANDROID_HOME ({}) does not appear to have valid Android SDK install
(cannot find build-tools)'''
raise HostError(msg.format(env.android_home))
versions = os.listdir(build_tools_directory)
for version in reversed(sorted(versions)):
aapt_path = os.path.join(build_tools_directory, version, 'aapt')
if os.path.isfile(aapt_path):
logger.debug('Using aapt for version {}'.format(version))
env.aapt = aapt_path
break
else:
raise HostError('aapt not found. Please make sure at least one Android '
'platform is installed.')
def _check_env():
global android_home, platform_tools, adb, aapt # pylint: disable=W0603
if not android_home:
android_home = os.getenv('ANDROID_HOME')
if android_home:
_env = _initialize_with_android_home(_AndroidEnvironment())
else:
_env = _initialize_without_android_home(_AndroidEnvironment())
android_home = _env.android_home
platform_tools = _env.platform_tools
adb = _env.adb
aapt = _env.aapt

552
devlib/utils/misc.py Normal file
View File

@@ -0,0 +1,552 @@
# Copyright 2013-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Miscellaneous functions that don't fit anywhere else.
"""
from __future__ import division
import os
import sys
import re
import string
import threading
import signal
import subprocess
import pkgutil
import logging
import random
from operator import itemgetter
from itertools import groupby
from functools import partial
# ABI --> architectures list
ABI_MAP = {
'armeabi': ['armeabi', 'armv7', 'armv7l', 'armv7el', 'armv7lh'],
'arm64': ['arm64', 'armv8', 'arm64-v8a', 'aarch64'],
}
# Vendor ID --> CPU part ID --> CPU variant ID --> Core Name
# None means variant is not used.
CPU_PART_MAP = {
0x41: { # ARM
0x926: {None: 'ARM926'},
0x946: {None: 'ARM946'},
0x966: {None: 'ARM966'},
0xb02: {None: 'ARM11MPCore'},
0xb36: {None: 'ARM1136'},
0xb56: {None: 'ARM1156'},
0xb76: {None: 'ARM1176'},
0xc05: {None: 'A5'},
0xc07: {None: 'A7'},
0xc08: {None: 'A8'},
0xc09: {None: 'A9'},
0xc0f: {None: 'A15'},
0xc14: {None: 'R4'},
0xc15: {None: 'R5'},
0xc20: {None: 'M0'},
0xc21: {None: 'M1'},
0xc23: {None: 'M3'},
0xc24: {None: 'M4'},
0xc27: {None: 'M7'},
0xd03: {None: 'A53'},
0xd07: {None: 'A57'},
0xd08: {None: 'A72'},
},
0x4e: { # Nvidia
0x0: {None: 'Denver'},
},
0x51: { # Qualcomm
0x02d: {None: 'Scorpion'},
0x04d: {None: 'MSM8960'},
0x06f: { # Krait
0x2: 'Krait400',
0x3: 'Krait450',
},
},
0x56: { # Marvell
0x131: {
0x2: 'Feroceon 88F6281',
}
},
}
def get_cpu_name(implementer, part, variant):
part_data = CPU_PART_MAP.get(implementer, {}).get(part, {})
if None in part_data: # variant does not determine core Name for this vendor
name = part_data[None]
else:
name = part_data.get(variant)
return name
def preexec_function():
# Ignore the SIGINT signal by setting the handler to the standard
# signal handler SIG_IGN.
signal.signal(signal.SIGINT, signal.SIG_IGN)
# Change process group in case we have to kill the subprocess and all of
# its children later.
# TODO: this is Unix-specific; would be good to find an OS-agnostic way
# to do this in case we wanna port WA to Windows.
os.setpgrp()
check_output_logger = logging.getLogger('check_output')
# Defined here rather than in devlib.exceptions due to module load dependencies
class TimeoutError(Exception):
"""Raised when a subprocess command times out. This is basically a ``WAError``-derived version
of ``subprocess.CalledProcessError``, the thinking being that while a timeout could be due to
programming error (e.g. not setting long enough timers), it is often due to some failure in the
environment, and there fore should be classed as a "user error"."""
def __init__(self, command, output):
super(TimeoutError, self).__init__('Timed out: {}'.format(command))
self.command = command
self.output = output
def __str__(self):
return '\n'.join([self.message, 'OUTPUT:', self.output or ''])
def check_output(command, timeout=None, ignore=None, inputtext=None, **kwargs):
"""This is a version of subprocess.check_output that adds a timeout parameter to kill
the subprocess if it does not return within the specified time."""
# pylint: disable=too-many-branches
if ignore is None:
ignore = []
elif isinstance(ignore, int):
ignore = [ignore]
elif not isinstance(ignore, list) and ignore != 'all':
message = 'Invalid value for ignore parameter: "{}"; must be an int or a list'
raise ValueError(message.format(ignore))
if 'stdout' in kwargs:
raise ValueError('stdout argument not allowed, it will be overridden.')
def callback(pid):
try:
check_output_logger.debug('{} timed out; sending SIGKILL'.format(pid))
os.killpg(pid, signal.SIGKILL)
except OSError:
pass # process may have already terminated.
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
stdin=subprocess.PIPE,
preexec_fn=preexec_function, **kwargs)
if timeout:
timer = threading.Timer(timeout, callback, [process.pid, ])
timer.start()
try:
output, error = process.communicate(inputtext)
finally:
if timeout:
timer.cancel()
retcode = process.poll()
if retcode:
if retcode == -9: # killed, assume due to timeout callback
raise TimeoutError(command, output='\n'.join([output, error]))
elif ignore != 'all' and retcode not in ignore:
raise subprocess.CalledProcessError(retcode, command, output='\n'.join([output, error]))
return output, error
def walk_modules(path):
"""
Given package name, return a list of all modules (including submodules, etc)
in that package.
"""
root_mod = __import__(path, {}, {}, [''])
mods = [root_mod]
for _, name, ispkg in pkgutil.iter_modules(root_mod.__path__):
submod_path = '.'.join([path, name])
if ispkg:
mods.extend(walk_modules(submod_path))
else:
submod = __import__(submod_path, {}, {}, [''])
mods.append(submod)
return mods
def ensure_directory_exists(dirpath):
"""A filter for directory paths to ensure they exist."""
if not os.path.isdir(dirpath):
os.makedirs(dirpath)
return dirpath
def ensure_file_directory_exists(filepath):
"""
A filter for file paths to ensure the directory of the
file exists and the file can be created there. The file
itself is *not* going to be created if it doesn't already
exist.
"""
ensure_directory_exists(os.path.dirname(filepath))
return filepath
def merge_dicts(*args, **kwargs):
if not len(args) >= 2:
raise ValueError('Must specify at least two dicts to merge.')
func = partial(_merge_two_dicts, **kwargs)
return reduce(func, args)
def _merge_two_dicts(base, other, list_duplicates='all', match_types=False, # pylint: disable=R0912,R0914
dict_type=dict, should_normalize=True, should_merge_lists=True):
"""Merge dicts normalizing their keys."""
merged = dict_type()
base_keys = base.keys()
other_keys = other.keys()
norm = normalize if should_normalize else lambda x, y: x
base_only = []
other_only = []
both = []
union = []
for k in base_keys:
if k in other_keys:
both.append(k)
else:
base_only.append(k)
union.append(k)
for k in other_keys:
if k in base_keys:
union.append(k)
else:
union.append(k)
other_only.append(k)
for k in union:
if k in base_only:
merged[k] = norm(base[k], dict_type)
elif k in other_only:
merged[k] = norm(other[k], dict_type)
elif k in both:
base_value = base[k]
other_value = other[k]
base_type = type(base_value)
other_type = type(other_value)
if (match_types and (base_type != other_type) and
(base_value is not None) and (other_value is not None)):
raise ValueError('Type mismatch for {} got {} ({}) and {} ({})'.format(k, base_value, base_type,
other_value, other_type))
if isinstance(base_value, dict):
merged[k] = _merge_two_dicts(base_value, other_value, list_duplicates, match_types, dict_type)
elif isinstance(base_value, list):
if should_merge_lists:
merged[k] = _merge_two_lists(base_value, other_value, list_duplicates, dict_type)
else:
merged[k] = _merge_two_lists([], other_value, list_duplicates, dict_type)
elif isinstance(base_value, set):
merged[k] = norm(base_value.union(other_value), dict_type)
else:
merged[k] = norm(other_value, dict_type)
else: # Should never get here
raise AssertionError('Unexpected merge key: {}'.format(k))
return merged
def merge_lists(*args, **kwargs):
if not len(args) >= 2:
raise ValueError('Must specify at least two lists to merge.')
func = partial(_merge_two_lists, **kwargs)
return reduce(func, args)
def _merge_two_lists(base, other, duplicates='all', dict_type=dict): # pylint: disable=R0912
"""
Merge lists, normalizing their entries.
parameters:
:base, other: the two lists to be merged. ``other`` will be merged on
top of base.
:duplicates: Indicates the strategy of handling entries that appear
in both lists. ``all`` will keep occurrences from both
lists; ``first`` will only keep occurrences from
``base``; ``last`` will only keep occurrences from
``other``;
.. note:: duplicate entries that appear in the *same* list
will never be removed.
"""
if not isiterable(base):
base = [base]
if not isiterable(other):
other = [other]
if duplicates == 'all':
merged_list = []
for v in normalize(base, dict_type) + normalize(other, dict_type):
if not _check_remove_item(merged_list, v):
merged_list.append(v)
return merged_list
elif duplicates == 'first':
base_norm = normalize(base, dict_type)
merged_list = normalize(base, dict_type)
for v in base_norm:
_check_remove_item(merged_list, v)
for v in normalize(other, dict_type):
if not _check_remove_item(merged_list, v):
if v not in base_norm:
merged_list.append(v) # pylint: disable=no-member
return merged_list
elif duplicates == 'last':
other_norm = normalize(other, dict_type)
merged_list = []
for v in normalize(base, dict_type):
if not _check_remove_item(merged_list, v):
if v not in other_norm:
merged_list.append(v)
for v in other_norm:
if not _check_remove_item(merged_list, v):
merged_list.append(v)
return merged_list
else:
raise ValueError('Unexpected value for list duplicates argument: {}. '.format(duplicates) +
'Must be in {"all", "first", "last"}.')
def _check_remove_item(the_list, item):
"""Helper function for merge_lists that implements checking wether an items
should be removed from the list and doing so if needed. Returns ``True`` if
the item has been removed and ``False`` otherwise."""
if not isinstance(item, basestring):
return False
if not item.startswith('~'):
return False
actual_item = item[1:]
if actual_item in the_list:
del the_list[the_list.index(actual_item)]
return True
def normalize(value, dict_type=dict):
"""Normalize values. Recursively normalizes dict keys to be lower case,
no surrounding whitespace, underscore-delimited strings."""
if isinstance(value, dict):
normalized = dict_type()
for k, v in value.iteritems():
key = k.strip().lower().replace(' ', '_')
normalized[key] = normalize(v, dict_type)
return normalized
elif isinstance(value, list):
return [normalize(v, dict_type) for v in value]
elif isinstance(value, tuple):
return tuple([normalize(v, dict_type) for v in value])
else:
return value
def convert_new_lines(text):
""" Convert new lines to a common format. """
return text.replace('\r\n', '\n').replace('\r', '\n')
def escape_quotes(text):
"""Escape quotes, and escaped quotes, in the specified text."""
return re.sub(r'\\("|\')', r'\\\\\1', text).replace('\'', '\\\'').replace('\"', '\\\"')
def escape_single_quotes(text):
"""Escape single quotes, and escaped single quotes, in the specified text."""
return re.sub(r'\\("|\')', r'\\\\\1', text).replace('\'', '\'\\\'\'')
def escape_double_quotes(text):
"""Escape double quotes, and escaped double quotes, in the specified text."""
return re.sub(r'\\("|\')', r'\\\\\1', text).replace('\"', '\\\"')
def getch(count=1):
"""Read ``count`` characters from standard input."""
if os.name == 'nt':
import msvcrt # pylint: disable=F0401
return ''.join([msvcrt.getch() for _ in xrange(count)])
else: # assume Unix
import tty # NOQA
import termios # NOQA
fd = sys.stdin.fileno()
old_settings = termios.tcgetattr(fd)
try:
tty.setraw(sys.stdin.fileno())
ch = sys.stdin.read(count)
finally:
termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)
return ch
def isiterable(obj):
"""Returns ``True`` if the specified object is iterable and
*is not a string type*, ``False`` otherwise."""
return hasattr(obj, '__iter__') and not isinstance(obj, basestring)
def as_relative(path):
"""Convert path to relative by stripping away the leading '/' on UNIX or
the equivant on other platforms."""
path = os.path.splitdrive(path)[1]
return path.lstrip(os.sep)
def get_cpu_mask(cores):
"""Return a string with the hex for the cpu mask for the specified core numbers."""
mask = 0
for i in cores:
mask |= 1 << i
return '0x{0:x}'.format(mask)
def which(name):
"""Platform-independent version of UNIX which utility."""
if os.name == 'nt':
paths = os.getenv('PATH').split(os.pathsep)
exts = os.getenv('PATHEXT').split(os.pathsep)
for path in paths:
testpath = os.path.join(path, name)
if os.path.isfile(testpath):
return testpath
for ext in exts:
testpathext = testpath + ext
if os.path.isfile(testpathext):
return testpathext
return None
else: # assume UNIX-like
try:
return check_output(['which', name])[0].strip() # pylint: disable=E1103
except subprocess.CalledProcessError:
return None
_bash_color_regex = re.compile('\x1b\\[[0-9;]+m')
def strip_bash_colors(text):
return _bash_color_regex.sub('', text)
def get_random_string(length):
"""Returns a random ASCII string of the specified length)."""
return ''.join(random.choice(string.ascii_letters + string.digits) for _ in xrange(length))
class LoadSyntaxError(Exception):
def __init__(self, message, filepath, lineno):
super(LoadSyntaxError, self).__init__(message)
self.filepath = filepath
self.lineno = lineno
def __str__(self):
message = 'Syntax Error in {}, line {}:\n\t{}'
return message.format(self.filepath, self.lineno, self.message)
RAND_MOD_NAME_LEN = 30
BAD_CHARS = string.punctuation + string.whitespace
TRANS_TABLE = string.maketrans(BAD_CHARS, '_' * len(BAD_CHARS))
def to_identifier(text):
"""Converts text to a valid Python identifier by replacing all
whitespace and punctuation."""
return re.sub('_+', '_', text.translate(TRANS_TABLE))
def unique(alist):
"""
Returns a list containing only unique elements from the input list (but preserves
order, unlike sets).
"""
result = []
for item in alist:
if item not in result:
result.append(item)
return result
def ranges_to_list(ranges_string):
"""Converts a sysfs-style ranges string, e.g. ``"0,2-4"``, into a list ,e.g ``[0,2,3,4]``"""
values = []
for rg in ranges_string.split(','):
if '-' in rg:
first, last = map(int, rg.split('-'))
values.extend(xrange(first, last + 1))
else:
values.append(int(rg))
return values
def list_to_ranges(values):
"""Converts a list, e.g ``[0,2,3,4]``, into a sysfs-style ranges string, e.g. ``"0,2-4"``"""
range_groups = []
for _, g in groupby(enumerate(values), lambda (i, x): i - x):
range_groups.append(map(itemgetter(1), g))
range_strings = []
for group in range_groups:
if len(group) == 1:
range_strings.append(str(group[0]))
else:
range_strings.append('{}-{}'.format(group[0], group[-1]))
return ','.join(range_strings)
def list_to_mask(values, base=0x0):
"""Converts the specified list of integer values into
a bit mask for those values. Optinally, the list can be
applied to an existing mask."""
for v in values:
base |= (1 << v)
return base
def mask_to_list(mask):
"""Converts the specfied integer bitmask into a list of
indexes of bits that are set in the mask."""
size = len(bin(mask)) - 2 # because of "0b"
return [size - i - 1 for i in xrange(size)
if mask & (1 << size - i - 1)]
__memo_cache = {}
def memoized(func):
"""A decorator for memoizing functions and methods."""
func_id = repr(func)
def memoize_wrapper(*args, **kwargs):
id_string = func_id + ','.join([str(id(a)) for a in args])
id_string += ','.join('{}={}'.format(k, v)
for k, v in kwargs.iteritems())
if id_string not in __memo_cache:
__memo_cache[id_string] = func(*args, **kwargs)
return __memo_cache[id_string]
return memoize_wrapper

107
devlib/utils/serial_port.py Normal file
View File

@@ -0,0 +1,107 @@
# Copyright 2013-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import time
from contextlib import contextmanager
from logging import Logger
import serial
import fdpexpect
# Adding pexpect exceptions into this module's namespace
from pexpect import EOF, TIMEOUT # NOQA pylint: disable=W0611
from devlib.exception import HostError
def pulse_dtr(conn, state=True, duration=0.1):
"""Set the DTR line of the specified serial connection to the specified state
for the specified duration (note: the initial state of the line is *not* checked."""
conn.setDTR(state)
time.sleep(duration)
conn.setDTR(not state)
def get_connection(timeout, init_dtr=None, logcls=Logger,
*args, **kwargs):
if init_dtr is not None:
kwargs['dsrdtr'] = True
try:
conn = serial.Serial(*args, **kwargs)
except serial.SerialException as e:
raise HostError(e.message)
if init_dtr is not None:
conn.setDTR(init_dtr)
conn.nonblocking()
conn.flushOutput()
target = fdpexpect.fdspawn(conn.fileno(), timeout=timeout)
target.logfile_read = logcls('read')
target.logfile_send = logcls('send')
# Monkey-patching sendline to introduce a short delay after
# chacters are sent to the serial. If two sendline s are issued
# one after another the second one might start putting characters
# into the serial device before the first one has finished, causing
# corruption. The delay prevents that.
tsln = target.sendline
def sendline(x):
tsln(x)
time.sleep(0.1)
target.sendline = sendline
return target, conn
def write_characters(conn, line, delay=0.05):
"""Write a single line out to serial charcter-by-character. This will ensure that nothing will
be dropped for longer lines."""
line = line.rstrip('\r\n')
for c in line:
conn.send(c)
time.sleep(delay)
conn.sendline('')
@contextmanager
def open_serial_connection(timeout, get_conn=False, init_dtr=None,
logcls=Logger, *args, **kwargs):
"""
Opens a serial connection to a device.
:param timeout: timeout for the fdpexpect spawn object.
:param conn: ``bool`` that specfies whether the underlying connection
object should be yielded as well.
:param init_dtr: specifies the initial DTR state stat should be set.
All arguments are passed into the __init__ of serial.Serial. See
pyserial documentation for details:
http://pyserial.sourceforge.net/pyserial_api.html#serial.Serial
:returns: a pexpect spawn object connected to the device.
See: http://pexpect.sourceforge.net/pexpect.html
"""
target, conn = get_connection(timeout, init_dtr=init_dtr,
logcls=logcls, *args, **kwargs)
if get_conn:
yield target, conn
else:
yield target
target.close() # Closes the file descriptor used by the conn.
del conn

261
devlib/utils/ssh.py Normal file
View File

@@ -0,0 +1,261 @@
# Copyright 2014-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import os
import stat
import logging
import subprocess
import re
import threading
import tempfile
import shutil
import pxssh
from pexpect import EOF, TIMEOUT, spawn
from devlib.exception import HostError, TargetError, TimeoutError
from devlib.utils.misc import which, strip_bash_colors, escape_single_quotes, check_output
ssh = None
scp = None
sshpass = None
logger = logging.getLogger('ssh')
def ssh_get_shell(host, username, password=None, keyfile=None, port=None, timeout=10, telnet=False):
_check_env()
if telnet:
if keyfile:
raise ValueError('keyfile may not be used with a telnet connection.')
conn = TelnetConnection()
else: # ssh
conn = pxssh.pxssh()
try:
if keyfile:
conn.login(host, username, ssh_key=keyfile, port=port, login_timeout=timeout)
else:
conn.login(host, username, password, port=port, login_timeout=timeout)
except EOF:
raise TargetError('Could not connect to {}; is the host name correct?'.format(host))
return conn
class TelnetConnection(pxssh.pxssh):
# pylint: disable=arguments-differ
def login(self, server, username, password='', original_prompt=r'[#$]', login_timeout=10,
auto_prompt_reset=True, sync_multiplier=1):
cmd = 'telnet -l {} {}'.format(username, server)
spawn._spawn(self, cmd) # pylint: disable=protected-access
i = self.expect('(?i)(?:password)', timeout=login_timeout)
if i == 0:
self.sendline(password)
i = self.expect([original_prompt, 'Login incorrect'], timeout=login_timeout)
else:
raise pxssh.ExceptionPxssh('could not log in: did not see a password prompt')
if i:
raise pxssh.ExceptionPxssh('could not log in: password was incorrect')
if not self.sync_original_prompt(sync_multiplier):
self.close()
raise pxssh.ExceptionPxssh('could not synchronize with original prompt')
if auto_prompt_reset:
if not self.set_unique_prompt():
self.close()
message = 'could not set shell prompt (recieved: {}, expected: {}).'
raise pxssh.ExceptionPxssh(message.format(self.before, self.PROMPT))
return True
def check_keyfile(keyfile):
"""
keyfile must have the right access premissions in order to be useable. If the specified
file doesn't, create a temporary copy and set the right permissions for that.
Returns either the ``keyfile`` (if the permissions on it are correct) or the path to a
temporary copy with the right permissions.
"""
desired_mask = stat.S_IWUSR | stat.S_IRUSR
actual_mask = os.stat(keyfile).st_mode & 0xFF
if actual_mask != desired_mask:
tmp_file = os.path.join(tempfile.gettempdir(), os.path.basename(keyfile))
shutil.copy(keyfile, tmp_file)
os.chmod(tmp_file, desired_mask)
return tmp_file
else: # permissions on keyfile are OK
return keyfile
class SshConnection(object):
default_password_prompt = '[sudo] password'
max_cancel_attempts = 5
@property
def name(self):
return self.host
def __init__(self,
host,
username,
password=None,
keyfile=None,
port=None,
timeout=10,
telnet=False,
password_prompt=None,
):
self.host = host
self.username = username
self.password = password
self.keyfile = check_keyfile(keyfile) if keyfile else keyfile
self.port = port
self.lock = threading.Lock()
self.password_prompt = password_prompt if password_prompt is not None else self.default_password_prompt
logger.debug('Logging in {}@{}'.format(username, host))
self.conn = ssh_get_shell(host, username, password, self.keyfile, port, timeout, telnet)
def push(self, source, dest, timeout=30):
dest = '{}@{}:{}'.format(self.username, self.host, dest)
return self._scp(source, dest, timeout)
def pull(self, source, dest, timeout=30):
source = '{}@{}:{}'.format(self.username, self.host, source)
return self._scp(source, dest, timeout)
def execute(self, command, timeout=None, check_exit_code=True, as_root=False, strip_colors=True):
with self.lock:
output = self._execute_and_wait_for_prompt(command, timeout, as_root, strip_colors)
if check_exit_code:
exit_code_text = self._execute_and_wait_for_prompt('echo $?', strip_colors=strip_colors, log=False)
try:
exit_code = int(exit_code_text.split()[0])
if exit_code:
message = 'Got exit code {}\nfrom: {}\nOUTPUT: {}'
raise TargetError(message.format(exit_code, command, output))
except (ValueError, IndexError):
logger.warning('Could not get exit code for "{}",\ngot: "{}"'.format(command, exit_code_text))
return output
def background(self, command, stdout=subprocess.PIPE, stderr=subprocess.PIPE):
port_string = '-p {}'.format(self.port) if self.port else ''
keyfile_string = '-i {}'.format(self.keyfile) if self.keyfile else ''
command = '{} {} {} {}@{} {}'.format(ssh, keyfile_string, port_string, self.username, self.host, command)
logger.debug(command)
if self.password:
command = _give_password(self.password, command)
return subprocess.Popen(command, stdout=stdout, stderr=stderr, shell=True)
def close(self):
logger.debug('Logging out {}@{}'.format(self.username, self.host))
self.conn.logout()
def cancel_running_command(self):
# simulate impatiently hitting ^C until command prompt appears
logger.debug('Sending ^C')
for _ in xrange(self.max_cancel_attempts):
self.conn.sendline(chr(3))
if self.conn.prompt(0.1):
return True
return False
def _execute_and_wait_for_prompt(self, command, timeout=None, as_root=False, strip_colors=True, log=True):
self.conn.prompt(0.1) # clear an existing prompt if there is one.
if as_root:
command = "sudo -- sh -c '{}'".format(escape_single_quotes(command))
if log:
logger.debug(command)
self.conn.sendline(command)
if self.password:
index = self.conn.expect_exact([self.password_prompt, TIMEOUT], timeout=0.5)
if index == 0:
self.conn.sendline(self.password)
else: # not as_root
if log:
logger.debug(command)
self.conn.sendline(command)
timed_out = self._wait_for_prompt(timeout)
# the regex removes line breaks potential introduced when writing
# command to shell.
output = process_backspaces(self.conn.before)
output = re.sub(r'\r([^\n])', r'\1', output)
if '\r\n' in output: # strip the echoed command
output = output.split('\r\n', 1)[1]
if timed_out:
self.cancel_running_command()
raise TimeoutError(command, output)
if strip_colors:
output = strip_bash_colors(output)
return output
def _wait_for_prompt(self, timeout=None):
if timeout:
return not self.conn.prompt(timeout)
else: # cannot timeout; wait forever
while not self.conn.prompt(1):
pass
return False
def _scp(self, source, dest, timeout=30):
# NOTE: the version of scp in Ubuntu 12.04 occasionally (and bizarrely)
# fails to connect to a device if port is explicitly specified using -P
# option, even if it is the default port, 22. To minimize this problem,
# only specify -P for scp if the port is *not* the default.
port_string = '-P {}'.format(self.port) if (self.port and self.port != 22) else ''
keyfile_string = '-i {}'.format(self.keyfile) if self.keyfile else ''
command = '{} -r {} {} {} {}'.format(scp, keyfile_string, port_string, source, dest)
pass_string = ''
logger.debug(command)
if self.password:
command = _give_password(self.password, command)
try:
check_output(command, timeout=timeout, shell=True)
except subprocess.CalledProcessError as e:
raise subprocess.CalledProcessError(e.returncode, e.cmd.replace(pass_string, ''), e.output)
except TimeoutError as e:
raise TimeoutError(e.command.replace(pass_string, ''), e.output)
def _give_password(password, command):
if not sshpass:
raise HostError('Must have sshpass installed on the host in order to use password-based auth.')
pass_string = "sshpass -p '{}' ".format(password)
return pass_string + command
def _check_env():
global ssh, scp, sshpass # pylint: disable=global-statement
if not ssh:
ssh = which('ssh')
scp = which('scp')
sshpass = which('sshpass')
if not (ssh and scp):
raise HostError('OpenSSH must be installed on the host.')
def process_backspaces(text):
chars = []
for c in text:
if c == chr(8) and chars: # backspace
chars.pop()
else:
chars.append(c)
return ''.join(chars)

113
devlib/utils/types.py Normal file
View File

@@ -0,0 +1,113 @@
# Copyright 2014-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Routines for doing various type conversions. These usually embody some higher-level
semantics than are present in standard Python types (e.g. ``boolean`` will convert the
string ``"false"`` to ``False``, where as non-empty strings are usually considered to be
``True``).
A lot of these are intened to stpecify type conversions declaratively in place like
``Parameter``'s ``kind`` argument. These are basically "hacks" around the fact that Python
is not the best language to use for configuration.
"""
import math
from devlib.utils.misc import isiterable, to_identifier, ranges_to_list, list_to_mask
def identifier(text):
"""Converts text to a valid Python identifier by replacing all
whitespace and punctuation."""
return to_identifier(text)
def boolean(value):
"""
Returns bool represented by the value. This is different from
calling the builtin bool() in that it will interpret string representations.
e.g. boolean('0') and boolean('false') will both yield False.
"""
false_strings = ['', '0', 'n', 'no', 'off']
if isinstance(value, basestring):
value = value.lower()
if value in false_strings or 'false'.startswith(value):
return False
return bool(value)
def integer(value):
"""Handles conversions for string respresentations of binary, octal and hex."""
if isinstance(value, basestring):
return int(value, 0)
else:
return int(value)
def numeric(value):
"""
Returns the value as number (int if possible, or float otherwise), or
raises ``ValueError`` if the specified ``value`` does not have a straight
forward numeric conversion.
"""
if isinstance(value, int):
return value
try:
fvalue = float(value)
except ValueError:
raise ValueError('Not numeric: {}'.format(value))
if not math.isnan(fvalue) and not math.isinf(fvalue):
ivalue = int(fvalue)
if ivalue == fvalue: # yeah, yeah, I know. Whatever. This is best-effort.
return ivalue
return fvalue
class caseless_string(str):
"""
Just like built-in Python string except case-insensitive on comparisons. However, the
case is preserved otherwise.
"""
def __eq__(self, other):
if isinstance(other, basestring):
other = other.lower()
return self.lower() == other
def __ne__(self, other):
return not self.__eq__(other)
def __cmp__(self, other):
if isinstance(basestring, other):
other = other.lower()
return cmp(self.lower(), other)
def format(self, *args, **kwargs):
return caseless_string(super(caseless_string, self).format(*args, **kwargs))
def bitmask(value):
if isinstance(value, basestring):
value = ranges_to_list(value)
if isiterable(value):
value = list_to_mask(value)
if not isinstance(value, int):
raise ValueError(value)
return value

116
devlib/utils/uboot.py Normal file
View File

@@ -0,0 +1,116 @@
#
# Copyright 2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import re
import time
import logging
from devlib.utils.serial_port import TIMEOUT
logger = logging.getLogger('U-Boot')
class UbootMenu(object):
"""
Allows navigating Das U-boot menu over serial (it relies on a pexpect connection).
"""
option_regex = re.compile(r'^\[(\d+)\]\s+([^\r]+)\r\n', re.M)
prompt_regex = re.compile(r'^([^\r\n]+):\s*', re.M)
invalid_regex = re.compile(r'Invalid input \(max (\d+)\)', re.M)
load_delay = 1 # seconds
default_timeout = 60 # seconds
def __init__(self, conn, start_prompt='Hit any key to stop autoboot'):
"""
:param conn: A serial connection as returned by ``pexect.spawn()``.
:param prompt: U-Boot menu prompt
:param start_prompt: The starting prompt to wait for during ``open()``.
"""
self.conn = conn
self.conn.crlf = '\n\r' # TODO: this has *got* to be a bug in U-Boot...
self.start_prompt = start_prompt
self.options = {}
self.prompt = None
def open(self, timeout=default_timeout):
"""
"Open" the UEFI menu by sending an interrupt on STDIN after seeing the
starting prompt (configurable upon creation of the ``UefiMenu`` object.
"""
self.conn.expect(self.start_prompt, timeout)
self.conn.sendline('')
time.sleep(self.load_delay)
self.conn.readline() # garbage
self.conn.sendline('')
self.prompt = self.conn.readline().strip()
def getenv(self):
output = self.enter('printenv')
result = {}
for line in output.split('\n'):
if '=' in line:
variable, value = line.split('=', 1)
result[variable.strip()] = value.strip()
return result
def setenv(self, variable, value, force=False):
force_str = ' -f' if force else ''
if value is not None:
command = 'setenv{} {} {}'.format(force_str, variable, value)
else:
command = 'setenv{} {}'.format(force_str, variable)
return self.enter(command)
def boot(self):
self.write_characters('boot')
def nudge(self):
"""Send a little nudge to ensure there is something to read. This is useful when you're not
sure if all out put from the serial has been read already."""
self.enter('')
def enter(self, value, delay=load_delay):
"""Like ``select()`` except no resolution is performed -- the value is sent directly
to the serial connection."""
# Empty the buffer first, so that only response to the input about to
# be sent will be processed by subsequent commands.
value = str(value)
self.empty_buffer()
self.write_characters(value)
self.conn.expect(self.prompt, timeout=delay)
return self.conn.before
def write_characters(self, line):
line = line.rstrip('\r\n')
for c in line:
self.conn.send(c)
time.sleep(0.05)
self.conn.sendline('')
def empty_buffer(self):
try:
while True:
time.sleep(0.1)
self.conn.read_nonblocking(size=1024, timeout=0.1)
except TIMEOUT:
pass
self.conn.buffer = ''

239
devlib/utils/uefi.py Normal file
View File

@@ -0,0 +1,239 @@
# Copyright 2014-2015 ARM Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import re
import time
import logging
from copy import copy
from devlib.utils.serial_port import write_characters, TIMEOUT
from devlib.utils.types import boolean
logger = logging.getLogger('UEFI')
class UefiConfig(object):
def __init__(self, config_dict):
if isinstance(config_dict, UefiConfig):
self.__dict__ = copy(config_dict.__dict__)
else:
try:
self.image_name = config_dict['image_name']
self.image_args = config_dict['image_args']
self.fdt_support = boolean(config_dict['fdt_support'])
except KeyError as e:
raise ValueError('Missing mandatory parameter for UEFI entry config: "{}"'.format(e))
self.initrd = config_dict.get('initrd')
self.fdt_path = config_dict.get('fdt_path')
if self.fdt_path and not self.fdt_support:
raise ValueError('FDT path has been specfied for UEFI entry, when FDT support is "False"')
class UefiMenu(object):
"""
Allows navigating UEFI menu over serial (it relies on a pexpect connection).
"""
option_regex = re.compile(r'^\[(\d+)\]\s+([^\r]+)\r\n', re.M)
prompt_regex = re.compile(r'^(\S[^\r\n]+):\s*', re.M)
invalid_regex = re.compile(r'Invalid input \(max (\d+)\)', re.M)
load_delay = 1 # seconds
default_timeout = 60 # seconds
def __init__(self, conn, prompt='The default boot selection will start in'):
"""
:param conn: A serial connection as returned by ``pexect.spawn()``.
:param prompt: The starting prompt to wait for during ``open()``.
"""
self.conn = conn
self.start_prompt = prompt
self.options = {}
self.prompt = None
self.attempting_invalid_retry = False
def wait(self, timeout=default_timeout):
"""
"Open" the UEFI menu by sending an interrupt on STDIN after seeing the
starting prompt (configurable upon creation of the ``UefiMenu`` object.
"""
self.conn.expect(self.start_prompt, timeout)
self.connect()
def connect(self, timeout=default_timeout):
self.nudge()
time.sleep(self.load_delay)
self.read_menu(timeout=timeout)
def create_entry(self, name, config):
"""Create a new UEFI entry using the parameters. The menu is assumed
to be at the top level. Upon return, the menu will be at the top level."""
logger.debug('Creating UEFI entry {}'.format(name))
self.nudge()
self.select('Boot Manager')
self.select('Add Boot Device Entry')
self.select('NOR Flash')
self.enter(config.image_name)
self.enter('y' if config.fdt_support else 'n')
if config.initrd:
self.enter('y')
self.enter(config.initrd)
else:
self.enter('n')
self.enter(config.image_args)
self.enter(name)
if config.fdt_path:
self.select('Update FDT path')
self.enter(config.fdt_path)
self.select('Return to main menu')
def delete_entry(self, name):
"""Delete the specified UEFI entry. The menu is assumed
to be at the top level. Upon return, the menu will be at the top level."""
logger.debug('Removing UEFI entry {}'.format(name))
self.nudge()
self.select('Boot Manager')
self.select('Remove Boot Device Entry')
self.select(name)
self.select('Return to main menu')
def select(self, option, timeout=default_timeout):
"""
Select the specified option from the current menu.
:param option: Could be an ``int`` index of the option, or a string/regex to
match option text against.
:param timeout: If a non-``int`` option is specified, the option list may need
need to be parsed (if it hasn't been already), this may block
and the timeout is used to cap that , resulting in a ``TIMEOUT``
exception.
:param delay: A fixed delay to wait after sending the input to the serial connection.
This should be set if input this action is known to result in a
long-running operation.
"""
if isinstance(option, basestring):
option = self.get_option_index(option, timeout)
self.enter(option)
def enter(self, value, delay=load_delay):
"""Like ``select()`` except no resolution is performed -- the value is sent directly
to the serial connection."""
# Empty the buffer first, so that only response to the input about to
# be sent will be processed by subsequent commands.
value = str(value)
self._reset()
write_characters(self.conn, value)
# TODO: in case the value is long an complicated, things may get
# screwed up (e.g. there may be line breaks injected), additionally,
# special chars might cause regex to fail. To avoid these issues i'm
# only matching against the first 5 chars of the value. This is
# entirely arbitrary and I'll probably have to find a better way of
# doing this at some point.
self.conn.expect(value[:5], timeout=delay)
time.sleep(self.load_delay)
def read_menu(self, timeout=default_timeout):
"""Parse serial output to get the menu options and the following prompt."""
attempting_timeout_retry = False
self.attempting_invalid_retry = False
while True:
index = self.conn.expect([self.option_regex, self.prompt_regex, self.invalid_regex, TIMEOUT],
timeout=timeout)
match = self.conn.match
if index == 0: # matched menu option
self.options[match.group(1)] = match.group(2)
elif index == 1: # matched prompt
self.prompt = match.group(1)
break
elif index == 2: # matched invalid selection
# We've sent an invalid input (which includes an empty line) at
# the top-level menu. To get back the menu options, it seems we
# need to enter what the error reports as the max + 1, so...
if not self.attempting_invalid_retry:
self.attempting_invalid_retry = True
val = int(match.group(1))
self.empty_buffer()
self.enter(val)
self.select('Return to main menu')
self.attempting_invalid_retry = False
else: # OK, that didn't work; panic!
raise RuntimeError('Could not read menu entries stuck on "{}" prompt'.format(self.prompt))
elif index == 3: # timed out
if not attempting_timeout_retry:
attempting_timeout_retry = True
self.nudge()
else: # Didn't help. Run away!
raise RuntimeError('Did not see a valid UEFI menu.')
else:
raise AssertionError('Unexpected response waiting for UEFI menu') # should never get here
def list_options(self, timeout=default_timeout):
"""Returns the menu index of the specified option text (uses regex matching). If the option
is not in the current menu, ``LookupError`` will be raised."""
if not self.prompt:
self.read_menu(timeout)
return self.options.items()
def get_option_index(self, text, timeout=default_timeout):
"""Returns the menu index of the specified option text (uses regex matching). If the option
is not in the current menu, ``LookupError`` will be raised."""
if not self.prompt:
self.read_menu(timeout)
for k, v in self.options.iteritems():
if re.search(text, v):
return k
raise LookupError(text)
def has_option(self, text, timeout=default_timeout):
"""Returns ``True`` if at least one of the options in the current menu has
matched (using regex) the specified text."""
try:
self.get_option_index(text, timeout)
return True
except LookupError:
return False
def nudge(self):
"""Send a little nudge to ensure there is something to read. This is useful when you're not
sure if all out put from the serial has been read already."""
self.enter('')
def empty_buffer(self):
"""Read everything from the serial and clear the internal pexpect buffer. This ensures
that the next ``expect()`` call will time out (unless further input will be sent to the
serial beforehand. This is used to create a "known" state and avoid unexpected matches."""
try:
while True:
time.sleep(0.1)
self.conn.read_nonblocking(size=1024, timeout=0.1)
except TIMEOUT:
pass
self.conn.buffer = ''
def _reset(self):
self.options = {}
self.prompt = None
self.empty_buffer()