Commit Graph

674 Commits

Author SHA1 Message Date
Eric Callahan 070e03d741 file_manager: Don't filter inotify events on the gcodes root
Broadcast events for all files and let clients determine
what should be shown.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-10-10 11:33:33 -04:00
Eric Callahan 7f1c327f02 file_manager: hidden folder bugfix
Since inotify supports hidden folders we shouldn't skip them when scanning for nodes.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-10-10 11:04:59 -04:00
Eric Callahan 0c311aeef4 mqtt: add support for publishing klipper status updates
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-10-08 14:17:13 -04:00
danijoo c3f1b290f8 power: add RF transmitter support
Signed-off: Daniel Bauer <github@dbauer.me>
2021-10-03 09:05:12 -04:00
Eric Callahan 86764657cf machine: add moonraker-telegram-bot to list of allowed services
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-09-29 18:08:08 -04:00
Eric Callahan fdfab66951 metadata: add PrusaSlicer aliases
The varous forks of PrusaSlicer are processed identically
outside of their identifier, so there is no need to
implement new classes for each fork.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-09-28 06:14:24 -04:00
Eric Callahan 9c09ca2ed5 update_manager: add support for moved git repos
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-09-24 12:22:45 -04:00
Eric Callahan d6fafef902 proc_stats: report websocket connection count
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-09-21 14:19:01 -04:00
Eric Callahan b1d739e032 moonraker: report websocket connection via /server/info
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-09-21 11:30:25 -04:00
Eric Callahan e76e83d8c1 moonraker: configure the tornado http client to use the curl backend
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-09-21 08:32:24 -04:00
Eric Callahan e92da4c82c power: add gpiod support for Arch Linux
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-09-15 18:09:20 -04:00
Eric Callahan 776e0d6c90 file_manager: strip leading slashes supplied in the upload path argument
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-09-14 14:43:00 -04:00
Stefan Dej 81176a6d31 confighelper: update 'Invalid config option' warning for better understanding
Signed-off-by: Stefan Dej <meteyou@gmail.com>
2021-08-29 14:15:12 -04:00
Eric Callahan 245053434f authorization: specifically identify basic auth
Raise an exception when a request with Basic Auth is received, however do not log the username/password.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-08-28 08:54:40 -04:00
Eric Callahan dfb8da6e3e authorization: improve error message for invalid auth headers
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-08-27 06:11:55 -04:00
Eric Callahan b040640813 update_manager: broadcast update_refreshed notification in the status refresh handler
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-08-26 15:44:26 -04:00
Eric Callahan 8a7b348899 update_manager: fix apt-get issue with changed release info
Add --allow-releaseinfo-change to the call to apt-get.  This resolves an issue where the repos are chaning their "Suite" value from stable to oldstable with the release of Debian Bullseye.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-08-24 06:39:54 -04:00
Eric Callahan 5204fe3e08 metadata: fix est time parsing for Simplify3D
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-08-07 11:45:44 -04:00
Eric Callahan 86e0ef0d5e git_deploy: use "git pull --rebase" when repo debug is enabled
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-27 05:52:56 -04:00
Eric Callahan 42f61ceafa power: attempt to retry failed device init for 120 seconds
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-18 14:26:10 -04:00
Eric Callahan c684b063b2 proc_stats: report network stats for all interfaces
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-18 08:49:38 -04:00
Eric Callahan 8f8d780c72 proc_stats: fix stat logging when the event loop is blocked
Convert the deque to a list before slicing.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-15 11:03:07 -04:00
Eric Callahan 91f264bdaf update_manager: fix unbound variable error
Signed-off-by:  Eric Callahan <arksine.code
2021-07-15 08:14:18 -04:00
Eric Callahan 1fdfb5299e moonraker: create a new asyncio event loop on restart
This should guarantee that no dangliing callbacks are leftover on a soft restart.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-15 06:57:54 -04:00
Eric Callahan 0d1431395e update_manager: fix potential race condition with auto_refresh
Wait until update manager init is complete before attempting starting the auto refresh periodic callback.  This prevents an auto refresh from blocking init if Moonraker is started during the auto refresh window.

Signed-off-by: Eric Callahan <arksine.code@gmail.com>
2021-07-15 06:52:04 -04:00
Eric Callahan 7f740ccb5a authorization: add warning if a domian in "cors_domains" contains a trailing slash
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 15:47:38 -04:00
Eric Callahan cb8d5ecb35 app_deploy: add "is_system_service" configuration option
This allows users to add update manager "clients" that don't have systemd services registered.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 07:47:43 -04:00
Eric Callahan 87444f5bcb zip_deploy: allow updates to stable releases when on the "beta" channel
Users on the "beta" channel should be updated to the latest release, including stable releases.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 07:47:43 -04:00
Eric Callahan 2ceafb0ff5 git_deploy: replace references to ioloop with eventloop
Wrap some additional system calls that could potentially block in "run_in_thread".

Signed-off-by: Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:34:15 -04:00
Eric Callahan 912840bbcd zip_deploy: replace references to ioloop with eventloop
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan d52554231a app_deploy: replace reference to ioloop with eventloop
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan c4796ee321 update_manager: replace references to ioloop with eventloop
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan 13a85fe9e0 file_manager: replace references to ioloop with eventloop
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan e4ecc1febe authorization: replace references to ioloop with eventloop
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan 646e97ad55 machine: replace reference to ioloop with eventloop
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan 05e2389dc1 mqtt: replace references to ioloop with eventloop
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan 8382b90682 paneldue: replace reference to ioloop with eventloop
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan c61cfa325f power: replace references of ioloop with EventLoop
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan 78e10cf7f8 proc_stats: Use the EventLoop to run blocking calls in in a background thread
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan 85d12da18d shell_command: replace tornado lock with asyncio lock
Also use asyncio's sleep method in place of tornado's wrapper around it.

Signed-off-by: Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan 4430d564fe websockets: replace references to ioloop with event loop
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan 8a3ff7a54a app: replace refrences to ioloop with eventloop
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan d224536552 moonraker: replace calls to ioloop with eventloop wrapper
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan 07cb64a191 eventloop: initial implementation
This adds a thin wrapper around asyncio's eventloop.  Generally it is similar to Tornado's IOLoop, however without the need to support all awaitables.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-12 06:16:18 -04:00
Eric Callahan fddc67db93 update_manager: add support for web betas
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-09 18:06:07 -04:00
Eric Callahan df43aca222 proc_stats: fix erroneous timer handle assignment
The delayed callback to "log_last_stats()" in the Watchdog shold not have its return value assigned to the watchdog handle.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-09 08:01:27 -04:00
Eric Callahan fb9206aa8f proc_stats: read system files in another thread
This should prevent the event loop from getting blocked by a system call.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-09 05:53:11 -04:00
Eric Callahan e20c12619b git_deploy: compare upstream url to lowercase url during validation
SIgned-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-08 12:09:17 -04:00
Eric Callahan 211c7e70e1 app_deploy: fix typo preventing repo clients from loading
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-08 11:58:11 -04:00
Eric Callahan dacd9f7565 update_manager: report detected_type and configured_type for applications
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-08 09:02:08 -04:00
Eric Callahan d76abe019f update_manager: return True when on update success
All calls to update now return a boolean value.  When performing a full upate this return value is used to
check if Moonraker should wait for Klippy to reconnect.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-08 07:55:50 -04:00
Eric Callahan b6cd19d6ef update_manager: make application package updates generic
This ignores an attempt to update packages if system updates are disabled and opens the possibility for supporting updates for other linux distributions.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-07 18:27:12 -04:00
Eric Callahan 74630f933a update_manager: use asyncio locks
Transion away from Tornado's wrappers, as their implementation of
"wait()" can be confusing.  Some of tornado's methods require that
you specify a delay in seconds, others requrire that you specify a
timeout relative to the Unix epoch.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-07 18:27:07 -04:00
Eric Callahan 9133b59dbf update_manager: add "full" update endpoint
This endpoint will perform a full system update.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-07 18:26:30 -04:00
Eric Callahan 10da7b714f update_manager: add support for zip deployment
This adds support for updating packaged applications from GitHub.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 908501cbc8 update_manager: remove unnecessary Is_refreshing check
The underlying deployment implementations use locks to prevent concurrent attempts at refeshing state, thus the "is_refreshing" check is redundant.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 5d35cc0d10 update_manager: Implement async streaming downloads
This allows Moonraker to report download progress.  This also resolves potential issues with I/O blocking the event loop.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan f2b48d0f1a update_manager: refactor github API requests
Save a request cache in the CommandHelper that stores the etag and value for each URL requested.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 88ee83c7d8 update_manager: use pathlib to represent file system paths
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan d3ec2dbae2 update_manager: refactor application deployment
This is the first step toward creating mutliple methods for deployment.  This patch creates a base class for all deployment objects and a base class for application deployment.  The git functionality has moved to its own module.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 78edcab86e confighelper: don't add supplemental configs to the original
Create a new ConfigParser that reads the supplemental and return a new ConfigWrapper.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 627db24c63 update_manager: make system updates optional
Remove the "distro" config option as it is not necessary.  If there is a need to identify the linux distribution that can now be done through the distro dependency.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 7c56a8fa74 utils: add method to validate packaged applications
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 0527904c48 moonraker: remove references to "system_args"
Rather than add these arguments to the config, use a method to access them from the server object.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 39a532cf87 utils: move git version parsing into its own method
This creates a generic utility for retreiving the git version.  Moonraker will now attempt to parse basic version info from a ".version" file if git describe fails.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan ebaac290e7 utils: add a method to calculate the hash of a directory
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 2ab63d75fc moonraker: fix imports for type checking
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 1c248d1de8 moonraker: remove check for component existence in "load_component()"
The importlib module will raise an exception if the component does not exist, so the check is redundant.  This allow allows packages to be loaded.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 261fbbc867 file_manager: move modules to its own package
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 346015e528 update_manager: move the "rebuild env" functionality
Initially rebuilding the venv was required to change env options to include dist packages.  This is no longer necessary so we can remove detection for it.  Building a venv could still be necessary, so move this functionality to its own method.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 8b8d538a9c update_manager: remove dist package options
These options are no longer necessary, as we now apply the dist path before the import in modules that require dist packages.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 5b260a5ba6 update_manager: correct relative paths
Update the paths to Moonraker and the supplemental configuration file.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>

Update update_manager.py
2021-07-06 19:25:10 -04:00
Arksine 03892a517e update_manger: move module into its own package
This includes its specific configuration file.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 30155c4a8c power: add special handling for gpiod import
Rather than symlink the gpiod dependency in a virtualenv, temporarily add the dist-package to sys.path then import gpiod.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine bfef2eb98e file_manager: update location of metadata script
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Arksine 7ccb455174 metadata: move extract_metadata.py to components
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:25:10 -04:00
Eric Callahan 80df137437 websockets: fix call to wait() in WebsocketHandler.close()
The wait timeout must be relative to the current time.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:24:52 -04:00
Eric Callahan a605cd3048 file_manager: NotifySyncLock bugfix
The time() method must be called on the current IOLoop.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 19:22:46 -04:00
Eric Callahan c6fee3c1f4 moonraker: send status eventtime as a second parameter to "notify_status_update"
This resolves a potential issue where clients could anticipate that all items in the status parameter are objects.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 14:56:18 -04:00
Eric Callahan 81854d1daf moonraker: report Klipper event time with status subscriptions
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
2021-07-06 14:35:09 -04:00
Eric Callahan 4836f52a5d shell_command: ignore utf8 decode errors
Also catch a potential KeyError when removing running commands.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-07-06 07:26:09 -04:00
Eric Callahan ecc0fbdae9 mqtt: return a boolean in the wait_connection method
Rather than require consumers handle the TimeoutError, handle it directly and return the connection state.

SIgned-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-28 12:45:50 -04:00
Eric Callahan 00f4bd594f app: prevent static file reads from blocking the event loop
Perform reads in a thread so File I/O does not block.

This patch also disables ETags for static files.  Tornado's default behavior of caching file hashes will not work as many of Moonraker's can be updated.  The previous workaround to this was to recalculate the checksum if the modified date changed.  This is inefficient  as its behavior is not much different than using "If-Modified-Since".

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-28 07:31:40 -04:00
Eric Callahan 3529acfeec proc_stats: report a blocked event loop
Use the PeriodicCallback in proc_stats to check if the event loop was blocked for a More than 5 seconds.  If this is detected, log
Moonraker's CPU usage over that time.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-27 17:10:49 -04:00
Eric Callahan 1510f66121 app: Process file uploads in a thread pool executor
This should prevent large file uploads from blocking the asyncio event loop for a long period of time.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-27 12:33:52 -04:00
Eric Callahan b6f9769488 websockets: refactor message handling
Implement a write buffer so that all calls to "write_message" are awaited.  This allows for more graceful shutdown if the websocket is closed.

When Moonraker shuts down, attempt to wait for all websockets to close before exiting.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-26 15:33:35 -04:00
Eric Callahan 0126aa794f mqtt: initial support for mqtt connections
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-26 08:10:32 -04:00
Eric Callahan 2bc7efd20e octoprint_compat: only register endpoints with the http transport
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-25 17:46:43 -04:00
Eric Callahan 566df69ca5 moonraker: expose additional app methods
Add references to the application's "get_websocket_manager" and "register_api_transport" methods.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-25 17:46:43 -04:00
Eric Callahan 40f21b10cd app: allow transport registration
This allows eligible components to register themselves as API transports.  By default the WebsocketManager is registered.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-25 17:46:43 -04:00
Eric Callahan a7801db6e7 websockets: Make the JsonRPC class more generic
Allow JsonRPC to be used with transports other than the websocket.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-25 17:46:43 -04:00
Eric Callahan 06bbca7ae3 moonraker: add method to set a component as failed
Some components require post-load initialization after the IOLoop has started.  Provide a method for these components to register a failure with the server.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-22 19:38:09 -04:00
Kamil Trzcinski 20673b9edd pause_resume: Make Moonraker to use `pause_resume/*` API
This switches Moonraker from calling `CANCEL_PRINT` and alikes
to rather use `pause_resume/` API that underneath calls
the relevant G-Codes.

This done, this way, allows Klipper to be able to interrupt
blocking operations gracefully.

Signed-off-by: Kamil Trzcinski <ayufan@ayufan.eu>
2021-06-15 12:22:43 -04:00
Rafael Schridi 3b4bfba85e Add variable domain support for Home Assistant
Defaults to the hard-coded "switch" domain used before

Signed-off-by:  Rafael Schridi <r.schridi@gmail.com>
2021-06-13 18:17:08 -04:00
Eric Callahan 42bdb42573 database: read out all namespaces on init
If a namespace contains an invalid key reading out the namespace should correct the issue.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-13 11:04:44 -04:00
Eric Callahan c9a958f861 database: track unsafe shutdowns
Keep track of the number of unsafe shutdowns a machine experiences.  This data is useful in diagnosing potential file system issues and issues with the datbase itself.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-13 09:15:09 -04:00
Eric Callahan 9a316fe84a database: check for invalid keys
It is invalid for a namespace to contain a top level key represented as an empty bytestring.  If this is detected, log the result and drop the invalid key.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-13 08:18:12 -04:00
Eric Callahan 92423911a9 file_manager: upload request fix
Make sure that inotify watches are added for new directories created via the upload endpoint.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-12 17:22:01 -04:00
Kamil Trzcinski 6770a965a6 history: add support for cancelled state
Signed-off-by: Kamil Trzcinski <ayufan@ayufan.eu>
2021-06-12 11:35:35 -04:00
Eric Callahan 9ea91f4f5d machine: report total memory from /proc/meminfo
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2021-06-10 19:58:11 -04:00