cardinal_pythonlib.tee

---

Original code copyright (C) 2009-2022 Rudolf Cardinal (rudolf@pobox.com).

This file is part of cardinal_pythonlib.

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

---

Support functions for “tee” functionality.

DEVELOPMENT NOTES

Initial failure:

• We can copy the Python logging output to a file; that’s part of the standard logging facility.
• We can also redirect our own stdout/stderr to a file and/or print a copy; that’s pretty easy to.
• We can manually capture subprocess stdout/stderr.
• We can redirect our own and subprocess stdout/stderr to a genuine file by duplicating the file descriptor(s): https://eli.thegreenplace.net/2015/redirecting-all-kinds-of-stdout-in-python/
• However, that file descriptor duplication method needs our file-like object to behave properly like a C-level file. That precludes the simpler kinds of “tee” behaviour in which a Python class pretends to be a file by implementing write(), close(), flush() methods.

So:

• redirect plain Python stderr/stdout
• handle subprocess stuff

See

• https://stackoverflow.com/questions/616645/how-do-i-duplicate-sys-stdout-to-a-log-file-in-python
• https://stackoverflow.com/questions/24931/how-to-capture-python-interpreters-and-or-cmd-exes-output-from-a-python-script
• https://www.python.org/dev/peps/pep-0343/
• https://stackoverflow.com/questions/4675728/redirect-stdout-to-a-file-in-python
• https://eli.thegreenplace.net/2015/redirecting-all-kinds-of-stdout-in-python/
• https://stackoverflow.com/questions/2996887/how-to-replicate-tee-behavior-in-python-when-using-subprocess
• https://stackoverflow.com/questions/4984428/python-subprocess-get-childrens-output-to-file-and-terminal/4985080#4985080

class cardinal_pythonlib.tee.TeeContextManager(file: TextIO, capture_stdout: bool = False, capture_stderr: bool = False)

Context manager to implement the function of the Unix tee command: that is, to save output to a file as well as display it to the console.

Note that this redirects Python’s sys.stdout or sys.stderr, but doesn’t redirect stdout/stderr from child processes – so use teed_call() to run them if you want those redirected too. See buildfunc.run() for an example.

Also, existing logs won’t be redirected (presumably because they’ve already taken a copy of their output streams); see tee_log() for an example of one way to manage this.

Parameters:

• file – file-like object to write to. We take a file object, not a filename, so we can apply multiple tee filters going to the same file.
• capture_stdout – capture stdout? Use this or capture_stderr
• capture_stderr – capture stderr? Use this or capture_stdout

We read the filename from file.name but this is purely cosmetic.

close() → None

To act as a file.

flush() → None

To act as a file.

write(message: str) → None

To act as a file.

cardinal_pythonlib.tee.tee(infile: IO, *files) → threading.Thread

Print the file-like object infile to the file-like object(s) files in a separate thread.

Starts and returns that thread.

The type (text, binary) must MATCH across all files.

From https://stackoverflow.com/questions/4984428/python-subprocess-get-childrens-output-to-file-and-terminal

A note on text versus binary IO:

TEXT files include:

• files opened in text mode ("r", "rt", "w", "wt")
• sys.stdin, sys.stdout
• io.StringIO(); see https://docs.python.org/3/glossary.html#term-text-file

BINARY files include:

• files opened in binary mode ("rb", "wb", "rb+"…)
• sys.stdin.buffer, sys.stdout.buffer
• io.BytesIO()
• gzip.GzipFile(); see https://docs.python.org/3/glossary.html#term-binary-file

$ python3  # don't get confused and use Python 2 by mistake!

t = open("/tmp/text.txt", "r+t")  # text mode is default
b = open("/tmp/bin.bin", "r+b")

t.write("hello\n")  # OK
# b.write("hello\n")  # raises TypeError

# t.write(b"world\n")  # raises TypeError
b.write(b"world\n")  # OK

t.flush()
b.flush()
t.seek(0)
b.seek(0)

x = t.readline()  # "hello\n"
y = b.readline()  # b"world\n"

cardinal_pythonlib.tee.tee_log(tee_file: TextIO, loglevel: int) → None

Context manager to add a file output stream to our logging system.

Parameters:

• tee_file – file-like object to write to
• loglevel – log level (e.g. logging.DEBUG) to use for this stream

cardinal_pythonlib.tee.teed_call(cmd_args, stdout_targets: List[TextIO] = None, stderr_targets: List[TextIO] = None, encoding: str = 'utf-8', **kwargs)

Runs a command and captures its output via tee() to one or more destinations. The output is always captured (otherwise we would lose control of the output and ability to tee it); if no destination is specified, we add a null handler.

We insist on TextIO output files to match sys.stdout (etc.).

A variation on: https://stackoverflow.com/questions/4984428/python-subprocess-get-childrens-output-to-file-and-terminal

Parameters:

• cmd_args – arguments for the command to run
• stdout_targets – file-like objects to write stdout to
• stderr_targets – file-like objects to write stderr to
• encoding – encoding to apply to stdout and stderr
• kwargs – additional arguments for subprocess.Popen