cardinal_pythonlib.tee
Original code copyright (C) 2009-2022 Rudolf Cardinal (rudolf@pobox.com).
This file is part of cardinal_pythonlib.
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Support functions for “tee” functionality.
DEVELOPMENT NOTES
Initial failure:
We can copy the Python logging output to a file; that’s part of the standard logging facility.
We can also redirect our own stdout/stderr to a file and/or print a copy; that’s pretty easy to.
We can manually capture subprocess stdout/stderr.
We can redirect our own and subprocess stdout/stderr to a genuine file by duplicating the file descriptor(s): https://eli.thegreenplace.net/2015/redirecting-all-kinds-of-stdout-in-python/
However, that file descriptor duplication method needs our file-like object to behave properly like a C-level file. That precludes the simpler kinds of “tee” behaviour in which a Python class pretends to be a file by implementing write(), close(), flush() methods.
So:
redirect plain Python stderr/stdout
handle subprocess stuff
See
https://stackoverflow.com/questions/616645/how-do-i-duplicate-sys-stdout-to-a-log-file-in-python
https://stackoverflow.com/questions/4675728/redirect-stdout-to-a-file-in-python
https://eli.thegreenplace.net/2015/redirecting-all-kinds-of-stdout-in-python/
- class cardinal_pythonlib.tee.TeeContextManager(file: TextIO, capture_stdout: bool = False, capture_stderr: bool = False)[source]
Context manager to implement the function of the Unix
tee
command: that is, to save output to a file as well as display it to the console.Note that this redirects Python’s
sys.stdout
orsys.stderr
, but doesn’t redirectstdout
/stderr
from child processes – so useteed_call()
to run them if you want those redirected too. Seebuildfunc.run()
for an example.Also, existing logs won’t be redirected (presumably because they’ve already taken a copy of their output streams); see
tee_log()
for an example of one way to manage this.- Parameters:
We read the filename from
file.name
but this is purely cosmetic.
- cardinal_pythonlib.tee.tee(infile: IO, *files: IO) Thread [source]
Print the file-like object
infile
to the file-like object(s)files
in a separate thread.Starts and returns that thread.
The type (text, binary) must MATCH across all files.
A note on text versus binary IO:
TEXT files include:
files opened in text mode (
"r"
,"rt"
,"w"
,"wt"
)sys.stdin
,sys.stdout
io.StringIO()
; see https://docs.python.org/3/glossary.html#term-text-file
BINARY files include:
files opened in binary mode (
"rb"
,"wb"
,"rb+"
…)sys.stdin.buffer
,sys.stdout.buffer
io.BytesIO()
gzip.GzipFile()
; see https://docs.python.org/3/glossary.html#term-binary-file
$ python3 # don't get confused and use Python 2 by mistake!
t = open("/tmp/text.txt", "r+t") # text mode is default b = open("/tmp/bin.bin", "r+b") t.write("hello\n") # OK # b.write("hello\n") # raises TypeError # t.write(b"world\n") # raises TypeError b.write(b"world\n") # OK t.flush() b.flush() t.seek(0) b.seek(0) x = t.readline() # "hello\n" y = b.readline() # b"world\n"
- cardinal_pythonlib.tee.tee_log(tee_file: TextIO, loglevel: int) None [source]
Context manager to add a file output stream to our logging system.
- cardinal_pythonlib.tee.teed_call(cmd_args, stdout_targets: List[TextIO] | None = None, stderr_targets: List[TextIO] | None = None, encoding: str = 'utf-8', **kwargs)[source]
Runs a command and captures its output via
tee()
to one or more destinations. The output is always captured (otherwise we would lose control of the output and ability totee
it); if no destination is specified, we add a null handler.We insist on
TextIO
output files to matchsys.stdout
(etc.).A variation on: https://stackoverflow.com/questions/4984428/python-subprocess-get-childrens-output-to-file-and-terminal