File Logger

Summary

Set up easily instantiated logging.Logger instances.

Create a few formatters and logging instances that can be easily imported and utilized across the package.

Currently stream_logger() is the easiest and most oft used entry point in this module.

Exceptions

NoUnNamedLoggers

Exception raised when a function in this module is called without a name argument for the logger.

Stream Logging

stream_logger(logger, log_level=logging.INFO, msg_format=None)

Returns a fully functional Logger instance for ready use.

File Logging

file_logger(filename, logger=None, shell=None, log_level=None, msg_format=None

Logging function that emits a logging.LogRecord to filename. Logger uses the following formatting by default.:

%(asctime)s : %(levelname)s : %(message)s

Parameters

• filename – str File to log a logging.LogRecord to.

• logger – logging.Logger, optional A logging.Logger instantiated in the calling module.

shellInteractiveShell, optional

Global instance of IPython. Can be None if not run in IPython though this hasn’t been tested.

log_levelint, optional

Level of log records.

msg_formatstr, optional

Representation of logging messages using parameters accepted by logging.Formatter. Uses standard % style string formatting.

Parameters

logger – logging.Logger instance

Raises

AssertionError – shell is not isinstance InteractiveShell.

JSON logger

Set up a logger that returns properly formatted JSON.

Parameters

loggerstr or logging.Logger, optional

Either a named Logger instance or the string representing the desired instance

json_formatterlogging.Formatter, optional JSONFormatter instance.

Included in the listed parameters to be explicit; however, it’s probably easier to not include the parameter as one is configured in the function anyway.

Returns

root_loggerlogging.Logger

Instance of a logging.Logger().

Remainder

This uses the IPython core IPython.core.logger.LoggingConfigurable to create a logging.handler.FileHandler that creates one new log file for every calendar day of the year.

Running IPython instances will continue to write to the log file if the session continues after the point when the logfile rolls over.

If multiple sessions are started, the new instances will append onto the same file as the older ones.

get_history

Extract a session from the IPython input history.

Usage

get_history(session_number)

Queries the IPython database for history entries.

Examples

python3 ipython-get-history.py 57 record.ipy

Summary for Logger

This script is a simple demonstration of IPython.core.history.HistoryAccessor. It should be possible to build much more flexible and powerful tools to browse and pull from the history database. This is a rewrite of that history accessor since IPython’s handling of history sessions out of the box is oddly limited.

Let’s ignore the direct calls to sys.argv and combine argparse and the magic_argparse functions to make something more durable and useful.

File Logger API

output to a file.

Todo

Logging TODOs

• Truncate output if it exceeds a certain threshold.

  • Run dir(np) or dir(pd) a couple of times and the logs become swamped.

• Possibly change that section under the shebang to also include 3 double quotes and in the comment add system info like py version, venv, conda, any of the 1000000 things you could add.

Note

Windows Users

Simulataneous IPython processes may throw.

PermissionError: [WinError 32]
The process cannot access the file because it is being used by another process:
'C:\\Users\\fac\\.ipython\\profile_default\\log\\log-2020-02-26.py' ->
'C:\\Users\\fac\\.ipython\\profile_default\\log\\log-2020-02-26.py.001~'

exception NoUnNamedLoggers(*args, **kwargs)

Bases: NotImplementedError

Raise this error if the logger a function was called with was anonymous.

__init__(*args, **kwargs)

print_history(hist_file=None)

Write the contents of the running shell’s sqlite history.

Parameters

hist_file (str (os.Pathlike), optional) – Path to history file. Assume default profile’s history.

get_history(session_number=None, raw=True, profile=None)

betterConfig(name=None, parent=None)

Similar to logging.basicConfig().

Parameters

• name (str, optional) – Name of the logger to return.

• parent (str, optional) – Name of the parent logger. I.E. One can provide an already initialized logger to this function and it will create a child logger. Defaults to None.

Returns

Anonymous Logger instance.

Return type

logging.Logger()

Notes

If a logging.Filter class is initialized with no args, it’s default behavior is to allow all logging.LogRecords to pass.

class ConfigurableLogger(**kwargs)

Bases: traitlets.config.configurable.LoggingConfigurable, logging.Logger

name = 'default_profile.startup.file_logger'

level = 30

__init__(logger=None, **kwargs)

logger: logging.Logger

log(msg, level=None)

Override the superclasses implementation of log.

Allow level to be a keyword argument.

Parameters

• msg (str) – Message to send to the logger

• level (int, optional) – Log level. Defaults to logging.WARNING or 30.

logging_decorator(f)

Create a decorator to be used around functions that need to be debugged.

Parameters

f (function) – The function to decorate

Notes

Calls a logger

ipython_logger()

Saves all commands run in the interactive namespace as valid IPython code.

Warning

This is not necessarily valid python code.

The commands are appended to a file in the directory of the profile in IPYTHONDIR or fallback to ~/.ipython. This file is named based on the date.

Parameters

shell (InteractiveShell) – Global IPython instance.

Raises

RuntimeError – If the configured logger is already logging to today’s date.

class MyHistoryAccessor(**kwargs)

Bases: IPython.core.history.HistoryAccessor

Modified HistoryAccessor to fetch the whole ipython history across all sessions

get_tail(n=10, raw=True, output=False, include_latest=False)

Get the last n lines from the history database.

Parameters

• n (int) – The number of lines to get

• raw (bool) – See get_range()

• output (bool) – See get_range()

• include_latest (bool) – If False (default), n+1 lines are fetched, and the latest one is discarded. This is intended to be used where the function is called by a user command, which it should not return.

Returns

Return type

Tuples as get_range()

rem_dups(lst)

access_all_history(*args)

stream_logger(logger, log_level=20, msg_format=None)

Set up a logging.Logger instance, add a stream handler.

Should do some validation on the log level there. There’s a really useful block of code in the tutorial.

Parameters

• logger (str) – Configure a passed logger. See example below.

• log_level (int, optional) – Level of log records. Defaults to 20.

• msg_format (str, optional) – Representation of logging messages. Uses standard % style string formatting. Defaults to %(asctime)s %(levelname)s %(message)s

Returns

logger – Defaults to logging.INFO and ‘%(asctime)s : %(levelname)s : %(message)s : ‘

Return type

logging.Logger() instance

Examples

>>> import logging
>>> from default_profile.startup.file_logger import stream_logger
>>> LOGGER = stream_logger(logging.getLogger(name=__name__))

file_logger(filename, logger=None, shell=None, log_level=20, msg_format=None)

json_logger(logger=None, json_formatter=None)

class JsonFormatter(fmt=None, datefmt=None, style='%', validate=True)

Bases: logging.Formatter

Return valid json for a configured handler.

format(record)

Format a logging.LogRecord() from an :exc:Exception.