Commit Graph

1129 Commits

Author SHA1 Message Date
Eric Callahan 5d35cc0d10 update_manager: Implement async streaming downloads
This allows Moonraker to report download progress.  This also resolves potential issues with I/O blocking the event loop.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan f2b48d0f1a update_manager: refactor github API requests
Save a request cache in the CommandHelper that stores the etag and value for each URL requested.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 88ee83c7d8 update_manager: use pathlib to represent file system paths
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan d3ec2dbae2 update_manager: refactor application deployment
This is the first step toward creating mutliple methods for deployment.  This patch creates a base class for all deployment objects and a base class for application deployment.  The git functionality has moved to its own module.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 78edcab86e confighelper: don't add supplemental configs to the original
Create a new ConfigParser that reads the supplemental and return a new ConfigWrapper.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 627db24c63 update_manager: make system updates optional
Remove the "distro" config option as it is not necessary.  If there is a need to identify the linux distribution that can now be done through the distro dependency.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 7c56a8fa74 utils: add method to validate packaged applications
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 0527904c48 moonraker: remove references to "system_args"
Rather than add these arguments to the config, use a method to access them from the server object.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 3642003dd2 workflows: add workflow to publish assets for a new release
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 2c47064d79 scripts: add release build scripts
Add a script that can build zipped releases to be distributed by Moonraker's Update Manager.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 39a532cf87 utils: move git version parsing into its own method
This creates a generic utility for retreiving the git version.  Moonraker will now attempt to parse basic version info from a ".version" file if git describe fails.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan ebaac290e7 utils: add a method to calculate the hash of a directory
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 5379e4b4bd scripts: remove version.txt
This file is no longer required for moonraker, the version info will either be stored in the pex file or retreived from git.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 6b03c4600d scripts: minor install script fixes
Fix the systemd unit description.  Make the launch command variable.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 7ed54a166b scripts: provide a log path option in install-moonraker.sh
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 2ab63d75fc moonraker: fix imports for type checking
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 1c248d1de8 moonraker: remove check for component existence in "load_component()"
The importlib module will raise an exception if the component does not exist, so the check is redundant.  This allow allows packages to be loaded.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 261fbbc867 file_manager: move modules to its own package
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 346015e528 update_manager: move the "rebuild env" functionality
Initially rebuilding the venv was required to change env options to include dist packages.  This is no longer necessary so we can remove detection for it.  Building a venv could still be necessary, so move this functionality to its own method.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 8b8d538a9c update_manager: remove dist package options
These options are no longer necessary, as we now apply the dist path before the import in modules that require dist packages.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 5b260a5ba6 update_manager: correct relative paths
Update the paths to Moonraker and the supplemental configuration file.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>

Update update_manager.py
2021-07-06 19:25:10 -04:00
Arksine 03892a517e update_manger: move module into its own package
This includes its specific configuration file.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 30155c4a8c power: add special handling for gpiod import
Rather than symlink the gpiod dependency in a virtualenv, temporarily add the dist-package to sys.path then import gpiod.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine bfef2eb98e file_manager: update location of metadata script
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 7ccb455174 metadata: move extract_metadata.py to components
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 80df137437 websockets: fix call to wait() in WebsocketHandler.close()
The wait timeout must be relative to the current time.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:24:52 -04:00
Eric Callahan a605cd3048 file_manager: NotifySyncLock bugfix
The time() method must be called on the current IOLoop.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:22:46 -04:00
Eric Callahan c7f8689bfe docs: fix incorrect documentation on eventtime
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 15:54:29 -04:00
Eric Callahan 12a7f957c3 docs: note addition of the "eventtime" param to "notify_status_update"
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 15:43:02 -04:00
Eric Callahan c6fee3c1f4 moonraker: send status eventtime as a second parameter to "notify_status_update"
This resolves a potential issue where clients could anticipate that all items in the status parameter are objects.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 14:56:18 -04:00
Eric Callahan 81854d1daf moonraker: report Klipper event time with status subscriptions
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
2021-07-06 14:35:09 -04:00
Eric Callahan 4836f52a5d shell_command: ignore utf8 decode errors
Also catch a potential KeyError when removing running commands.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 07:26:09 -04:00
Eric Callahan 5c05afae42 docs: Update mkdocs version
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-28 20:27:40 -04:00
Eric Callahan 77052e15b0 docs: update MQTT documentation
Reflect the new behavior of wait_connection()

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-28 20:27:40 -04:00
Eric Callahan ecc0fbdae9 mqtt: return a boolean in the wait_connection method
Rather than require consumers handle the TimeoutError, handle it directly and return the connection state.

SIgned-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-28 12:45:50 -04:00
Eric Callahan 00f4bd594f app: prevent static file reads from blocking the event loop
Perform reads in a thread so File I/O does not block.

This patch also disables ETags for static files.  Tornado's default behavior of caching file hashes will not work as many of Moonraker's can be updated.  The previous workaround to this was to recalculate the checksum if the modified date changed.  This is inefficient  as its behavior is not much different than using "If-Modified-Since".

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-28 07:31:40 -04:00
Eric Callahan 3529acfeec proc_stats: report a blocked event loop
Use the PeriodicCallback in proc_stats to check if the event loop was blocked for a More than 5 seconds.  If this is detected, log
Moonraker's CPU usage over that time.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-27 17:10:49 -04:00
Eric Callahan 1510f66121 app: Process file uploads in a thread pool executor
This should prevent large file uploads from blocking the asyncio event loop for a long period of time.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-27 12:33:52 -04:00
Eric Callahan c22dcc18dc docs: Remove unreferenced macro
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-26 20:12:10 -04:00
Eric Callahan b6f9769488 websockets: refactor message handling
Implement a write buffer so that all calls to "write_message" are awaited.  This allows for more graceful shutdown if the websocket is closed.

When Moonraker shuts down, attempt to wait for all websockets to close before exiting.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-26 15:33:35 -04:00
Eric Callahan 188dc4c782 docs: add MQTT documentation
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-26 11:44:29 -04:00
Eric Callahan 0126aa794f mqtt: initial support for mqtt connections
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-26 08:10:32 -04:00
Eric Callahan 2bc7efd20e octoprint_compat: only register endpoints with the http transport
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-25 17:46:43 -04:00
Eric Callahan 566df69ca5 moonraker: expose additional app methods
Add references to the application's "get_websocket_manager" and "register_api_transport" methods.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-25 17:46:43 -04:00
Eric Callahan 40f21b10cd app: allow transport registration
This allows eligible components to register themselves as API transports.  By default the WebsocketManager is registered.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-25 17:46:43 -04:00
Eric Callahan a7801db6e7 websockets: Make the JsonRPC class more generic
Allow JsonRPC to be used with transports other than the websocket.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-25 17:46:43 -04:00
Eric Callahan 06bbca7ae3 moonraker: add method to set a component as failed
Some components require post-load initialization after the IOLoop has started.  Provide a method for these components to register a failure with the server.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-22 19:38:09 -04:00
Eric Callahan 331be63c9e scripts: add paho-mqtt dependency
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-22 08:21:26 -04:00
Eric Callahan d3df568aec extract_metadata: don't remove ufp files after failed extraction
Cura writes ufp files in chunks, closing the file after each chunk is written.  This will result in a failed extraction, and removing the file will result in corruption.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-18 06:15:54 -04:00
Eric Callahan 9e909e8532 docs: add "cancelled" state to print_stats documentation
Refactor the available values so that they are more readable.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-16 15:50:04 -04:00