cardinal_pythonlib.winservice


Original code copyright (C) 2009-2022 Rudolf Cardinal (rudolf@pobox.com).

This file is part of cardinal_pythonlib.

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.


Functions to manage Windows services.

For an example, see crate_anon/tools/winservice.py

Details

Synopsis, applied to a service named CRATE

  • INSTALL: run this script with “install” argument, as administrator

  • RUN: use Windows service manager, or NET START CRATE

  • STOP: use Windows service manager, or NET STOP CRATE

  • STATUS: SC QUERY CRATE

  • DEBUG: run this script with “debug” argument

    • Log messages are sent to the console.
    • Press CTRL-C to abort.
  • If the debug script succeeds but the start command doesn’t…

    • Find the service in the Windows service manage.

    • Right-click it and inspect its properties. You’ll see the name of the actual program being run, e.g. D:\venvs\crate\Lib\site-packages\win32\pythonservice.exe

    • Try running this from the command line (outside any virtual environment).

    • In my case this failed with:

      pythonservice.exe - Unable to Locate Component
      This application has failed to start because pywintypes34.dll was not
      found. Re-installing the application may fix this problem.
      
    • That DLL was in: D:\venvs\crate\Lib\site-packages\pypiwin32_system32 … so add that to the system PATH… and reboot… and then it’s happy.

    • However, that’s not ideal for a virtual environment!

    • Looking at win32serviceutil.py, it seems we could do better by specifying _exe_name_ (to replace the default PythonService.exe) and _exe_args_. The sequence

      myscript.py install
      -> win32serviceutil.HandleCommandLine()
          ... fishes things out of cls._exe_name_, etc.
      -> win32serviceutil.InstallService()
          ... builds a command line
          ... by default:
          "d:\venvs\crate\lib\site-packages\win32\PythonService.exe"
      -> win32service.CreateService()
      
    • So how, in the normal situation, does PythonService.exe find our script?

    • At this point, see also https://stackoverflow.com/questions/34696815

    • Source code is: https://github.com/tjguk/pywin32/blob/master/win32/src/PythonService.cpp

    • Starting a service directly with PrepareToHostSingle:

    • SUCCESS! Method is:

      • In service class:

        _exe_name_ = sys.executable  # python.exe in the virtualenv
        _exe_args_ = '"{}"'.format(os.path.realpath(__file__))  # this script
        

      – In main:

      if len(sys.argv) == 1:
          try:
              print("Trying to start service directly...")
              evtsrc_dll = os.path.abspath(servicemanager.__file__)
              servicemanager.PrepareToHostSingle(CratewebService)  # CLASS
              servicemanager.Initialize('aservice', evtsrc_dll)
              servicemanager.StartServiceCtrlDispatcher()
          except win32service.error as details:
              print("Failed: {}".format(details))
              # print(repr(details.__dict__))
              errnum = details.winerror
              if errnum == winerror.ERROR_FAILED_SERVICE_CONTROLLER_CONNECT:
                  win32serviceutil.usage()
      else:
          win32serviceutil.HandleCommandLine(CratewebService)  # CLASS
      
    • Now, if you run it directly with no arguments, from a command prompt, it will fail and print the usage message, but if you run it from the service manager (with no arguments), it’ll start the service. Everything else seems to work. Continues to work when the PATH doesn’t include the virtual environment.

    • However, it breaks the “debug” option. The “debug” option reads the registry about the INSTALLED service to establish the name of the program that it runs. It assumes PythonService.exe.

Script parameters

If you run this script with no parameters, you’ll see this:

Usage: 'crate_windows_service-script.py [options] install|update|remove|start [...]|stop|restart [...]|debug [...]'
Options for 'install' and 'update' commands only:
 --username domain\username : The Username the service is to run under
 --password password : The password for the username
 --startup [manual|auto|disabled|delayed] : How the service starts, default = manual
 --interactive : Allow the service to interact with the desktop.
 --perfmonini file: .ini file to use for registering performance monitor data
 --perfmondll file: .dll file to use when querying the service for
   performance data, default = perfmondata.dll
Options for 'start' and 'stop' commands only:
 --wait seconds: Wait for the service to actually start or stop.
                 If you specify --wait with the 'stop' option, the service
                 and all dependent services will be stopped, each waiting
                 the specified period.

Windows functions:
    - CreateEvent: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682396(v=vs.85).aspx
    - WaitForSingleObject: https://msdn.microsoft.com/en-us/library/windows/desktop/ms687032(v=vs.85).aspx

Problems killing things

We had this:

def terminate(self):
    if not self.running:
        return
    if WINDOWS:
        # Under Windows, terminate() is an alias for kill(), which is
        # the hard kill. This is the soft kill:
        # https://docs.python.org/3.4/library/subprocess.html

        # SOMETHING GOES SERIOUSLY AWRY HERE.
        # - Tracebacks are from a slightly irrelevant place.
        # - Not all instances of OSError are caught.
        # - Running two processes, you get messages like:
        #       * process 1/2: sending CTRL-C
        #       * process 2/2: failed to send CTRL-C...
        #   without the other expected pair of messages.
        # DOES THIS MEAN A BUG IN SUBPROCESS?
        # Not sure. Removed "send_signal" code.

        # try:
        #     # Ctrl-C is generally "softer" than Ctrl-Break.
        #     # Specifically: CherryPy prints exiting message upon Ctrl-C
        #     # and just stops on Ctrl-Break.
        #     self.warning("Asking process to stop (sending CTRL-C)")
        #     self.process.send_signal(CTRL_C_EVENT)
        #     # self.warning("Asking process to stop (sending CTRL-BREAK)")
        #     # self.process.send_signal(CTRL_BREAK_EVENT)
        # except OSError:
        #     # In practice: "OSError: [WinError 6] The handle is invalid"
        #     self.warning("Failed to send CTRL-C; using hard kill")
        #     self.process.terminate()  # hard kill under Windows

        self.warning("Can't terminate nicely; using hard kill")
        self.process.terminate()  # hard kill under Windows

        # The PROBLEM is that Celery processes live on.
    else:
        self.warning("Asking process to stop (SIGTERM)")
        self.process.terminate()  # soft kill under POSIX

However, the CTRL-C/CTRL-BREAK method failed, and a hard kill left Celery stuff running (because it killed the root, not child, processes, I presume). Looked at django-windows-tools,

but it crashed in a print statement – Python 2 only at present (2016-05-11, version 0.1.1). However, its process management is instructive; it uses “multiprocessing”, not “subprocess”. The multiprocessing module calls Python functions. And its docs explicitly note that terminate() leaves descendant processes orphaned.

See in particular

Python bug?

Maybe a subprocess bug. Better luck with ctypes.windll.kernel32.GenerateConsoleCtrlEvent.

Current method tries a variety of things under Windows:

CTRL-C -> CTRL-BREAK -> TASKKILL /T -> TASKKILL /T /F -> kill()

… which are progressively less graceful in terms of child processes getting to clean up. Still, it works (usually at one of the two TASKKILL stages).

“The specified service is marked for deletion”

class cardinal_pythonlib.winservice.ProcessDetails(name: str, procargs: List[str], logfile_out: str = '', logfile_err: str = '')[source]

Description of a process.

Parameters:
  • name – cosmetic name of the process
  • procargs – command-line arguments
  • logfile_out – filename to write stdout to
  • logfile_err – filename to write stderr to
class cardinal_pythonlib.winservice.ProcessManager(details: cardinal_pythonlib.winservice.ProcessDetails, procnum: int, nprocs: int, kill_timeout_sec: float = 5, debugging: bool = False)[source]

Object that manages a single process.

Parameters:
  • details – description of the process as a ProcessDetails object
  • procnum – for cosmetic purposes only: the process sequence number of this process
  • nprocs – for cosmetic purposes only: the total number of processes (including others not managed by this instance)
  • kill_timeout_sec – how long (in seconds) will we wait for the process to end peacefully, before we try to kill it?
  • debugging – be verbose?
close_logs() → None[source]

Close Python disk logs.

debug(msg: str) → None[source]

If we are being verbose, write a debug message to the Python disk log.

error(msg: str) → None[source]

Write an error message to the Windows Application log (± to the Python disk log).

fullname

Description of the process.

info(msg: str) → None[source]

Write an info message to the Windows Application log (± to the Python disk log).

open_logs() → None[source]

Open Python disk logs.

start() → None[source]

Starts a subprocess. Optionally routes its output to our disk logs.

stop() → None[source]

Stops a subprocess.

Asks nicely. Waits. Asks less nicely. Repeat until subprocess is dead.

Todo

cardinal_pythonlib.winservice.ProcessManager.stop: make it reliable under Windows

wait(timeout_s: float = None) → int[source]

Wait for up to timeout_s for the child process to finish.

Parameters:timeout_s – maximum time to wait or None to wait forever
Returns:process return code; or 0 if it wasn’t running, or 1 if it managed to exit without a return code
Raises:subprocess.TimeoutExpired – if the process continues to run
warning(msg: str) → None[source]

Write a warning message to the Windows Application log (± to the Python disk log).

class cardinal_pythonlib.winservice.WindowsService(args: List[Any] = None)[source]

Class representing a Windows service.

Derived classes must set the following class properties:

# you can NET START/STOP the service by the following name
_svc_name_ = "CRATE"

# this text shows up as the service name in the Service
# Control Manager (SCM)
_svc_display_name_ = "CRATE web service"

# this text shows up as the description in the SCM
_svc_description_ = "Runs Django/Celery processes for CRATE web site"

# how to launch?
_exe_name_ = sys.executable  # python.exe in the virtualenv
_exe_args_ = '"{}"'.format(os.path.realpath(__file__))  # this script
SvcDoRun() → None[source]

Called when the service is started.

SvcStop() → None[source]

Called when the service is being shut down.

debug(msg: str) → None[source]

If we are being verbose, write a debug message to the Python log.

error(msg: str) → None[source]

Write an error message to the Windows Application log (± to the Python disk log).

info(msg: str) → None[source]

Write an info message to the Windows Application log (± to the Python disk log).

main() → None[source]

Main entry point. Runs service().

run_debug() → None[source]

Enable verbose mode and call main().

run_processes(procdetails: List[cardinal_pythonlib.winservice.ProcessDetails], subproc_run_timeout_sec: float = 1, stop_event_timeout_ms: int = 1000, kill_timeout_sec: float = 5) → None[source]

Run multiple child processes.

Parameters:
  • procdetails – list of ProcessDetails objects (q.v.)
  • subproc_run_timeout_sec – time (in seconds) to wait for each process when polling child processes to see how they’re getting on (default 1)
  • stop_event_timeout_ms – time to wait (in ms) while checking the Windows stop event for this service (default 1000)
  • kill_timeout_sec – how long (in seconds) will we wait for the subprocesses to end peacefully, before we try to kill them?

Todo

cardinal_pythonlib.winservice.WindowsService: NOT YET IMPLEMENTED: Windows service autorestart

service() → None[source]

Service function. Must be overridden by derived classes.

test_service(filename: str = 'C:\\test_win_svc.txt', period_ms: int = 5000) → None[source]

A test service.

Writes to a file occasionally, so you can see it’s running.

Parameters:
  • filename – file to write data to periodically
  • period_ms – period, in milliseconds
cardinal_pythonlib.winservice.generic_service_main(cls: Type[cardinal_pythonlib.winservice.WindowsService], name: str) → None[source]

Call this from your command-line entry point to manage a service.

  • Via inherited functions, enables you to install, update, remove, start, stop, and restart the service.
  • Via our additional code, allows you to run the service function directly from the command line in debug mode, using the debug command.
  • Run with an invalid command like help to see help (!).

See https://mail.python.org/pipermail/python-win32/2008-April/007299.html

Parameters:
  • cls – class deriving from WindowsService
  • name – name of this service