File IO¶
Reading and Writing Files¶
- can.LogReader(filename, **kwargs)[source]¶
Find and return the appropriate
MessageReader
instance for a given file suffix.- The format is determined from the file suffix which can be one of:
.asc
can.ASCReader
.blf
can.BLFReader
.csv
can.CSVReader
.db
can.SqliteReader
.mf4
can.MF4Reader
(optional, depends on asammdf).trc
can.TRCReader
Gzip compressed files can be used as long as the original files suffix is one of the above (e.g. filename.asc.gz).
Exposes a simple iterator interface, to use simply:
for msg in can.LogReader("some/path/to/my_file.log"): print(msg)
- Parameters:
filename (str | os.PathLike[str]) – the filename/path of the file to read from
kwargs (Any)
- Raises:
ValueError – if the filename’s suffix is of an unknown file type
- Return type:
Note
There are no time delays, if you want to reproduce the measured delays between messages look at the
can.MessageSync
class.Note
This function itself is just a dispatcher, and any positional and keyword arguments are passed on to the returned instance.
- can.Logger(filename, **kwargs)[source]¶
Find and return the appropriate
MessageWriter
instance for a given file suffix.- The format is determined from the file suffix which can be one of:
.asc
can.ASCWriter
.blf
can.BLFWriter
.csv:
can.CSVWriter
.db
can.SqliteWriter
.mf4
can.MF4Writer
(optional, depends on asammdf).trc
can.TRCWriter
.txt
can.Printer
Any of these formats can be used with gzip compression by appending the suffix .gz (e.g. filename.asc.gz). However, third-party tools might not be able to read these files.
The filename may also be None, to fall back to
can.Printer
.The log files may be incomplete until stop() is called due to buffering.
- Parameters:
- Raises:
ValueError – if the filename’s suffix is of an unknown file type
- Return type:
Note
This function itself is just a dispatcher, and any positional and keyword arguments are passed on to the returned instance.
- can.io.logger.MESSAGE_WRITERS: Final[Dict[str, Type[MessageWriter]]] = {'.asc': <class 'can.io.asc.ASCWriter'>, '.blf': <class 'can.io.blf.BLFWriter'>, '.csv': <class 'can.io.csv.CSVWriter'>, '.db': <class 'can.io.sqlite.SqliteWriter'>, '.log': <class 'can.io.canutils.CanutilsLogWriter'>, '.mf4': <class 'can.io.mf4.MF4Writer'>, '.trc': <class 'can.io.trc.TRCWriter'>, '.txt': <class 'can.io.printer.Printer'>}¶
A map of file suffixes to their corresponding
can.io.generic.MessageWriter
class
- can.io.player.MESSAGE_READERS: Final[Dict[str, Type[MessageReader]]] = {'.asc': <class 'can.io.asc.ASCReader'>, '.blf': <class 'can.io.blf.BLFReader'>, '.csv': <class 'can.io.csv.CSVReader'>, '.db': <class 'can.io.sqlite.SqliteReader'>, '.log': <class 'can.io.canutils.CanutilsLogReader'>, '.mf4': <class 'can.io.mf4.MF4Reader'>, '.trc': <class 'can.io.trc.TRCReader'>}¶
A map of file suffixes to their corresponding
can.io.generic.MessageReader
class
Printer¶
- class can.Printer(file=None, append=False, **kwargs)[source]¶
Bases:
MessageWriter
The Printer class is a subclass of
Listener
which simply prints any messages it receives to the terminal (stdout). A message is turned into a string using__str__()
.- Attr write_to_file:
True if this instance prints to a file instead of standard out
- Parameters:
file (TextIO | None) – An optional path-like object or a file-like object to “print” to instead of writing to standard out (stdout). If this is a file-like object, it has to be opened in text write mode, not binary write mode.
append (bool) – If set to True messages, are appended to the file, else the file is truncated
kwargs (Any)
CSVWriter¶
- class can.CSVWriter(file, append=False, **kwargs)[source]¶
Bases:
TextIOMessageWriter
Writes a comma separated text file with a line for each message. Includes a header line.
The columns are as follows:
name of column
format description
example
timestamp
decimal float
1483389946.197
arbitration_id
hex
0x00dadada
extended
1 == True, 0 == False
1
remote
1 == True, 0 == False
0
error
1 == True, 0 == False
0
dlc
int
6
data
base64 encoded
WzQyLCA5XQ==
Each line is terminated with a platform specific line separator.
- Parameters:
file (TextIO) – a path-like object or a file-like object to write to. If this is a file-like object, is has to open in text write mode, not binary write mode.
append (bool) – if set to True messages are appended to the file and no header line is written, else the file is truncated and starts with a newly written header line
kwargs (Any)
- class can.CSVReader(file, **kwargs)[source]¶
Bases:
TextIOMessageReader
Iterator over CAN messages from a .csv file that was generated by
CSVWriter
or that uses the same format as described there. Assumes that there is a header and thus skips the first line.Any line separator is accepted.
SqliteWriter¶
- class can.SqliteWriter(file, table_name='messages', **kwargs)[source]¶
Bases:
MessageWriter
,BufferedReader
Logs received CAN data to a simple SQL database.
The sqlite database may already exist, otherwise it will be created when the first message arrives.
Messages are internally buffered and written to the SQL file in a background thread. Ensures that all messages that are added before calling
stop()
are actually written to the database after that call returns. Thus, callingstop()
may take a while.- Attr str table_name:
the name of the database table used for storing the messages
- Attr int num_frames:
the number of frames actually written to the database, this excludes messages that are still buffered
- Attr float last_write:
the last time a message war actually written to the database, as given by
time.time()
- Parameters:
Note
When the listener’s
stop()
method is called the thread writing to the database will continue to receive and internally buffer messages if they continue to arrive before theGET_MESSAGE_TIMEOUT
.If the
GET_MESSAGE_TIMEOUT
expires before a message is received, the internal buffer is written out to the database file.However if the bus is still saturated with messages, the Listener will continue receiving until the
MAX_TIME_BETWEEN_WRITES
timeout is reached or more thanMAX_BUFFER_SIZE_BEFORE_WRITES
messages are buffered.Note
The database schema is given in the documentation of the loggers.
- Parameters:
Warning
In contrary to all other readers/writers the Sqlite handlers do not accept file-like objects as the file parameter.
- GET_MESSAGE_TIMEOUT = 0.25¶
Number of seconds to wait for messages from internal queue
- MAX_BUFFER_SIZE_BEFORE_WRITES = 500¶
Maximum number of messages to buffer before writing to the database
- MAX_TIME_BETWEEN_WRITES = 5.0¶
Maximum number of seconds to wait between writes to the database
- class can.SqliteReader(file, table_name='messages', **kwargs)[source]¶
Bases:
MessageReader
Reads recorded CAN messages from a simple SQL database.
This class can be iterated over or used to fetch all messages in the database with
read_all()
.Calling
len()
on this object might not run in constant time.- Attr str table_name:
the name of the database table used for storing the messages
- Parameters:
Note
The database schema is given in the documentation of the loggers.
- Parameters:
Warning
In contrary to all other readers/writers the Sqlite handlers do not accept file-like objects as the file parameter. It also runs in
append=True
mode all the time.- read_all()[source]¶
Fetches all messages in the database.
- Return type:
Generator[can.Message]
Database table format¶
The messages are written to the table messages
in the sqlite database
by default. The table is created if it does not already exist.
The entries are as follows:
Name |
Data type |
Note |
ts |
REAL |
The timestamp of the message |
arbitration_id |
INTEGER |
The arbitration id, might use the extended format |
extended |
INTEGER |
|
remote |
INTEGER |
|
error |
INTEGER |
|
dlc |
INTEGER |
The data length code (DLC) |
data |
BLOB |
The content of the message |
ASC (.asc Logging format)¶
ASCWriter logs CAN data to an ASCII log file compatible with other CAN tools such as Vector CANalyzer/CANoe and other. Since no official specification exists for the format, it has been reverse- engineered from existing log files. One description of the format can be found here.
Note
Channels will be converted to integers.
- class can.ASCWriter(file, channel=1, **kwargs)[source]¶
Bases:
TextIOMessageWriter
Logs CAN data to an ASCII log file (.asc).
The measurement starts with the timestamp of the first registered message. If a message has a timestamp smaller than the previous one or None, it gets assigned the timestamp that was written for the last message. It the first message does not have a timestamp, it is set to zero.
- Parameters:
ASCReader reads CAN data from ASCII log files .asc, as further references can-utils can be used: asc2log, log2asc.
- class can.ASCReader(file, base='hex', relative_timestamp=True, **kwargs)[source]¶
Bases:
TextIOMessageReader
Iterator of CAN messages from a ASC logging file. Meta data (comments, bus statistics, J1939 Transport Protocol messages) is ignored.
- Parameters:
file (TextIO) – a path-like object or as file-like object to read from If this is a file-like object, is has to opened in text read mode, not binary read mode.
base (str) – Select the base(hex or dec) of id and data. If the header of the asc file contains base information, this value will be overwritten. Default “hex”.
relative_timestamp (bool) – Select whether the timestamps are relative (starting at 0.0) or absolute (starting at the system time). Default True = relative.
kwargs (Any)
Log (.log can-utils Logging format)¶
CanutilsLogWriter logs CAN data to an ASCII log file compatible with can-utils <https://github.com/linux-can/can-utils> As specification following references can-utils can be used: asc2log, log2asc.
- class can.CanutilsLogWriter(file, channel='vcan0', append=False, **kwargs)[source]¶
Bases:
TextIOMessageWriter
Logs CAN data to an ASCII log file (.log). This class is is compatible with “candump -L”.
If a message has a timestamp smaller than the previous one (or 0 or None), it gets assigned the timestamp that was written for the last message. It the first message does not have a timestamp, it is set to zero.
- Parameters:
file (TextIO) – a path-like object or as file-like object to write to If this is a file-like object, is has to opened in text write mode, not binary write mode.
channel (str) – a default channel to use when the message does not have a channel set
append (bool) – if set to True messages are appended to the file, else the file is truncated
kwargs (Any)
CanutilsLogReader reads CAN data from ASCII log files .log
- class can.CanutilsLogReader(file, **kwargs)[source]¶
Bases:
TextIOMessageReader
Iterator over CAN messages from a .log Logging File (candump -L).
Note
.log-format looks for example like this:
(0.0) vcan0 001#8d00100100820100
BLF (Binary Logging Format)¶
Implements support for BLF (Binary Logging Format) which is a proprietary CAN log format from Vector Informatik GmbH.
The data is stored in a compressed format which makes it very compact.
Note
Channels will be converted to integers.
- class can.BLFWriter(file, append=False, channel=1, compression_level=-1, **kwargs)[source]¶
Bases:
FileIOMessageWriter
Logs CAN data to a Binary Logging File compatible with Vector’s tools.
- Parameters:
file (BinaryIO) – a path-like object or as file-like object to write to If this is a file-like object, is has to opened in mode “wb+”.
channel (int) – Default channel to log as if not specified by the interface.
append (bool) – Append messages to an existing log file.
compression_level (int) – An integer from 0 to 9 or -1 controlling the level of compression. 1 (Z_BEST_SPEED) is fastest and produces the least compression. 9 (Z_BEST_COMPRESSION) is slowest and produces the most. 0 means that data will be stored without processing. The default value is -1 (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default compromise between speed and compression (currently equivalent to level 6).
kwargs (Any)
- application_id = 5¶
Application identifier for the log writer
- log_event(text, timestamp=None)[source]¶
Add an arbitrary message to the log file as a global marker.
- max_container_size = 131072¶
Max log container size of uncompressed data
The following class can be used to read messages from BLF file:
- class can.BLFReader(file, **kwargs)[source]¶
Bases:
BinaryIOMessageReader
Iterator of CAN messages from a Binary Logging File.
Only CAN messages and error frames are supported. Other object types are silently ignored.
MF4 (Measurement Data Format v4)¶
Implements support for MF4 (Measurement Data Format v4) which is a proprietary format from ASAM (Association for Standardization of Automation and Measuring Systems), widely used in many automotive software (Vector CANape, ETAS INCA, dSPACE ControlDesk, etc.).
The data is stored in a compressed format which makes it compact.
Note
MF4 support has to be installed as an extra with for example pip install python-can[mf4]
.
Note
Channels will be converted to integers.
Note
MF4Writer does not suppport the append mode.
- class can.MF4Writer(file, database=None, compression_level=2, **kwargs)[source]¶
Bases:
BinaryIOMessageWriter
Logs CAN data to an ASAM Measurement Data File v4 (.mf4).
MF4Writer does not support append mode.
If a message has a timestamp smaller than the previous one or None, it gets assigned the timestamp that was written for the last message. It the first message does not have a timestamp, it is set to zero.
- Parameters:
file (BinaryIO | GzipFile) – A path-like object or as file-like object to write to. If this is a file-like object, is has to be opened in binary write mode, not text write mode.
database (str | os.PathLike[str] | None) – optional path to a DBC or ARXML file that contains message description.
compression_level (int) – compression option as integer (default 2) * 0 - no compression * 1 - deflate (slower, but produces smaller files) * 2 - transposition + deflate (slowest, but produces the smallest files)
kwargs (Any)
The MDF format is very flexible regarding the internal structure and it is used to handle data from multiple sources, not just CAN bus logging. MDF4Writer will always create a fixed internal file structure where there will be three channel groups (for standard, error and remote frames). Using this fixed file structure allows for a simple implementation of MDF4Writer and MF4Reader classes. Therefor MF4Reader can only replay files created with MF4Writer.
The following class can be used to read messages from MF4 file:
- class can.MF4Reader(file, **kwargs)[source]¶
Bases:
BinaryIOMessageReader
Iterator of CAN messages from a MF4 logging file.
The MF4Reader only supports MF4 files with CAN bus logging.
- Parameters:
TRC¶
Implements basic support for the TRC file format.
Note
Comments and contributions are welcome on what file versions might be relevant.
- class can.TRCWriter(file, channel=1, **kwargs)[source]¶
Bases:
TextIOMessageWriter
Logs CAN data to text file (.trc).
The measurement starts with the timestamp of the first registered message. If a message has a timestamp smaller than the previous one or None, it gets assigned the timestamp that was written for the last message. If the first message does not have a timestamp, it is set to zero.
- Parameters:
The following class can be used to read messages from TRC file:
- class can.TRCReader(file, **kwargs)[source]¶
Bases:
TextIOMessageReader
Iterator of CAN messages from a TRC logging file.
Rotating Loggers¶
- class can.io.BaseRotatingLogger(**kwargs)[source]¶
Bases:
MessageWriter
,ABC
Base class for rotating CAN loggers. This class is not meant to be instantiated directly. Subclasses must implement the
should_rollover()
anddo_rollover()
methods according to their rotation strategy.The rotation behavior can be further customized by the user by setting the
namer
androtator
attributes after instantiating the subclass.These attributes as well as the methods
rotation_filename()
androtate()
and the corresponding docstrings are carried over from the python builtinBaseRotatingHandler
.Subclasses must set the _writer attribute upon initialization.
- Parameters:
- namer: Callable[[str | os.PathLike[str]], str | os.PathLike[str]] | None = None¶
If this attribute is set to a callable, the
rotation_filename()
method delegates to this callable. The parameters passed to the callable are those passed torotation_filename()
.
- on_message_received(msg)[source]¶
This method is called to handle the given message.
- Parameters:
msg (Message) – the delivered message
- Return type:
None
- rotate(source, dest)[source]¶
When rotating, rotate the current log.
The default implementation calls the
rotator
attribute of the handler, if it’s callable, passing the source and dest arguments to it. If the attribute isn’t callable (the default isNone
), the source is simply renamed to the destination.
- rotation_filename(default_name)[source]¶
Modify the filename of a log file when rotating.
This is provided so that a custom filename can be provided. The default implementation calls the
namer
attribute of the handler, if it’s callable, passing the default name to it. If the attribute isn’t callable (the default isNone
), the name is returned unchanged.
- rotator: Callable[[str | os.PathLike[str], str | os.PathLike[str]], None] | None = None¶
If this attribute is set to a callable, the
rotate()
method delegates to this callable. The parameters passed to the callable are those passed torotate()
.
- stop()[source]¶
Stop handling new messages.
Carry out any final tasks to ensure data is persisted and cleanup any open resources.
- Return type:
None
- abstract property writer: FileIOMessageWriter¶
This attribute holds an instance of a writer class which manages the actual file IO.
- class can.SizedRotatingLogger(base_filename, max_bytes=0, **kwargs)[source]¶
Bases:
BaseRotatingLogger
Log CAN messages to a sequence of files with a given maximum size.
The logger creates a log file with the given base_filename. When the size threshold is reached the current log file is closed and renamed by adding a timestamp and the rollover count. A new log file is then created and written to.
This behavior can be customized by setting the
namer
androtator
attribute.Example:
from can import Notifier, SizedRotatingLogger from can.interfaces.vector import VectorBus bus = VectorBus(channel=[0], app_name="CANape", fd=True) logger = SizedRotatingLogger( base_filename="my_logfile.asc", max_bytes=5 * 1024 ** 2, # =5MB ) logger.rollover_count = 23 # start counter at 23 notifier = Notifier(bus=bus, listeners=[logger])
- The SizedRotatingLogger currently supports the formats
.asc:
can.ASCWriter
.blf
can.BLFWriter
.csv:
can.CSVWriter
.txt
can.Printer
(if pointing to a file)
Note
The
can.SqliteWriter
is not supported yet.The log files on disk may be incomplete due to buffering until
stop()
is called.- Parameters:
base_filename (str | os.PathLike[str]) – A path-like object for the base filename. The log file format is defined by the suffix of base_filename.
max_bytes (int) – The size threshold at which a new log file shall be created. If set to 0, no rollover will be performed.
kwargs (Any)
- property writer: FileIOMessageWriter¶
This attribute holds an instance of a writer class which manages the actual file IO.
Replaying Files¶
- class can.MessageSync(messages, timestamps=True, gap=0.0001, skip=60.0)[source]¶
Used to iterate over some given messages in the recorded time.
Creates an new MessageSync instance.
- Parameters:
messages (Iterable[Message]) – An iterable of
can.Message
instances.timestamps (bool) – Use the messages’ timestamps. If False, uses the gap parameter as the time between messages.
gap (float) – Minimum time between sent messages in seconds
skip (float) – Skip periods of inactivity greater than this (in seconds).
Example:
import can with can.LogReader("my_logfile.asc") as reader, can.Bus(interface="virtual") as bus: for msg in can.MessageSync(messages=reader): print(msg) bus.send(msg)