When a folder is copied processing can take several minutes, depending on the size and number of gcode files in the
path. Suppress all file events on a node until processing
is complete.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
The the cancel object post processor results in an additional
"create_file" notification when the old file is replaced.
Track previous "create_file" notifications for valid gcode
files so the dups can be suppressed.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
When destructively iterating over pending metdata requests
don't pop the pending item until metadata processing is
complete. If a call to `parse_metadata()` has a request
pending, return the asyncio Event associated with that
request.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Now that the update manager doesn't rely on the rate
limit it should always initialize successfully. An individual
updater may still fail to initialize, however this does
not prevent the update manager from loading.
Singed-off-by: Eric Callahan <arksine.code@gmail.com>
The rate limit fields can be extracted from the response
to an API request, even if that request fails with 403. This speeds up update manager initialization and reduces
failures if github is unreachable.
SIgned-off-by: Eric Callahan <arksine.code@gmail.com>
This returns the checksum of a config section and can be used to check
if the section has changed.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Make sure that the queue is set to ready after resuming,
even if the next job is unable to be loaded.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
When set to true, the queue will automatically transition
to the next job in the queue upon completion of the
current job. When false the queue will pause between
jobs, requiring that the user manually resume.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Resolve issues with ipv6 addresses. Only register the
net_state_changed event handler if Moonraker is bound
to all interfaces. Build a new instance of ServiceInfo
for each update rather than attempt to modify the
existing object.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Add `--plain` and `--no-legend` to the command to list service units.
This fixes a bug where a service could be prefixed with a unique
character, resulting undetected services.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Some devices on the Home Assistant server do not update their internal state immediately after a request returns. Introduce a configurable delay to work around this issue.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This provides corrective action in the event that an
invalid user entry makes its way into the database.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
For gpiod versions 1.3 or greater use the "default_val" keyword argument to set the default, otherwise use
"default_vals".
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Determine if a firmware restart should be postponed by checking Klippy's state. This should correctly schedule the restart regardless of whether or not the device is bound to Klipper.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This handles a specific scenario where a power device is bound to the Klipper service and is configured to perform a firmware restart after power on. In this case, we can detect if the firmware restart is required and abort it if necessary.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
When this option is set to "True", the device will power on after the file manager queues an upload for printing.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Now that init occurs before the server starts there is no need to block requests until post initialization.
Signed-off-buy: Eric Callahan <arksine.code@gmail.com>
Occasionally a server restart will fail as the new loop instantiates as closed. This seems to be an issue with asyncio, attempt to work around by retyring to create a new loop.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Previously a set was used to remove duplicate components, however this is unnecessary as the `load_component` method immediately returns dups. Using a list should preserve the load order based on the configuration, making it more predictable.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
The `send_event()` method now returns a future that can be awaited until all callbacks are complete. All events emitted during Klipper init are now awaited, and a Lock is used to prevent re-entry. This resolves potential timing issues with commands sent during the init sequence.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Components may call a "component_init" method for initialization that requires async calls. This allows moonraker to track init progress and guarantee init order.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Remove the "load_component_multi" method as it is not
necessary. Components are responsible for loading
"child" sections as required. Core components can
now have their own config section.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Add support for holding multiple printer jobs in a queue.
When a job successfully completes Moonraker will load
the next job in the queue.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This module adds a helper to track Klippy's "print_stats" module. It
stores the current stats and emits events for changes in job state.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
The `update_service_status()` method could result in the creation of multiple subprocesses for each update, resulting in a performance hit as subprocesses are inherently expensive.
Fetch the state for all detected services in a single subprocess.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Log all import errors other than ModuleNotFound errors. Add a server warning if a gpio type is configured and the gpiod module is not loaded.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This resolves an issue where a path is containter two slashes as a separator reports as outside of the root.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Most containers do not have access to systemd, so using
systemd-detect-virt will fail. We can check the cgroup
and sched files of the first process to reliably determine
if Moonraker is running within a container.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
It isn't valid to run some commands from within containers. Use systmd-detect-virt to detect a virtualized environment and report the type and id via the system_info endpoint.
If Moonraker is running from within a container do not allow access to the reboot and shutdown endpoints.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
When a symbolic link is created there is no completion event, so immediately fire the notification.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This resolves an issue where an allowed service is not
tagged as available if it is not loaded when moonraker
starts.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Stop polling Klippy after it it has exited the "startup" state.
When this has been detected, emit a "klippy_startup"
event.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
The varous forks of PrusaSlicer are processed identically
outside of their identifier, so there is no need to
implement new classes for each fork.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Raise an exception when a request with Basic Auth is received, however do not log the username/password.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Add --allow-releaseinfo-change to the call to apt-get. This resolves an issue where the repos are chaning their "Suite" value from stable to oldstable with the release of Debian Bullseye.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Wait until update manager init is complete before attempting starting the auto refresh periodic callback. This prevents an auto refresh from blocking init if Moonraker is started during the auto refresh window.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This adds a thin wrapper around asyncio's eventloop. Generally it is similar to Tornado's IOLoop, however without the need to support all awaitables.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
The delayed callback to "log_last_stats()" in the Watchdog shold not have its return value assigned to the watchdog handle.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
All calls to update now return a boolean value. When performing a full upate this return value is used to
check if Moonraker should wait for Klippy to reconnect.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This ignores an attempt to update packages if system updates are disabled and opens the possibility for supporting updates for other linux distributions.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Transion away from Tornado's wrappers, as their implementation of
"wait()" can be confusing. Some of tornado's methods require that
you specify a delay in seconds, others requrire that you specify a
timeout relative to the Unix epoch.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
The underlying deployment implementations use locks to prevent concurrent attempts at refeshing state, thus the "is_refreshing" check is redundant.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This allows Moonraker to report download progress. This also resolves potential issues with I/O blocking the event loop.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This is the first step toward creating mutliple methods for deployment. This patch creates a base class for all deployment objects and a base class for application deployment. The git functionality has moved to its own module.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Remove the "distro" config option as it is not necessary. If there is a need to identify the linux distribution that can now be done through the distro dependency.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Rather than add these arguments to the config, use a method to access them from the server object.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This creates a generic utility for retreiving the git version. Moonraker will now attempt to parse basic version info from a ".version" file if git describe fails.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
The importlib module will raise an exception if the component does not exist, so the check is redundant. This allow allows packages to be loaded.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Initially rebuilding the venv was required to change env options to include dist packages. This is no longer necessary so we can remove detection for it. Building a venv could still be necessary, so move this functionality to its own method.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
These options are no longer necessary, as we now apply the dist path before the import in modules that require dist packages.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Rather than symlink the gpiod dependency in a virtualenv, temporarily add the dist-package to sys.path then import gpiod.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This resolves a potential issue where clients could anticipate that all items in the status parameter are objects.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Rather than require consumers handle the TimeoutError, handle it directly and return the connection state.
SIgned-off-by: Eric Callahan <arksine.code@gmail.com>
Perform reads in a thread so File I/O does not block.
This patch also disables ETags for static files. Tornado's default behavior of caching file hashes will not work as many of Moonraker's can be updated. The previous workaround to this was to recalculate the checksum if the modified date changed. This is inefficient as its behavior is not much different than using "If-Modified-Since".
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Use the PeriodicCallback in proc_stats to check if the event loop was blocked for a More than 5 seconds. If this is detected, log
Moonraker's CPU usage over that time.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This should prevent large file uploads from blocking the asyncio event loop for a long period of time.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Implement a write buffer so that all calls to "write_message" are awaited. This allows for more graceful shutdown if the websocket is closed.
When Moonraker shuts down, attempt to wait for all websockets to close before exiting.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Add references to the application's "get_websocket_manager" and "register_api_transport" methods.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This allows eligible components to register themselves as API transports. By default the WebsocketManager is registered.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Some components require post-load initialization after the IOLoop has started. Provide a method for these components to register a failure with the server.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This switches Moonraker from calling `CANCEL_PRINT` and alikes
to rather use `pause_resume/` API that underneath calls
the relevant G-Codes.
This done, this way, allows Klipper to be able to interrupt
blocking operations gracefully.
Signed-off-by: Kamil Trzcinski <ayufan@ayufan.eu>
Keep track of the number of unsafe shutdowns a machine experiences. This data is useful in diagnosing potential file system issues and issues with the datbase itself.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
It is invalid for a namespace to contain a top level key represented as an empty bytestring. If this is detected, log the result and drop the invalid key.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Some slicers create a temporary file during export, then rename the file with gcode extension after completion. If file metadata does not exist during an attempt to move metadata, scan the file.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Shutdown the websocket and http server before doing final component cleanup. This prevents clients from making a request after components have been closed.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This removes the cryptography dependency in favor of libsodium. Also removed is python-jose, as we must generate our own JWTs for use with EdDSA.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
use libnacl instead of pynacl
Systemd units are case sensitive, do not convert the "name" argument for service requests to lower case.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Home Assistant returns an empty response when switching a switch
on or off. Will now requery the switch status and return it's result
Signed-off-by: Alan Harper <alan@aussiegeek.net>
Clients can use this in situations where a browser may
prompt the user to take action prior to sending the
request. After the user accepts, Moonraker will redirect the user to the url provided in the request.
SIgned-off-by: Eric Callahan <arksine.code@gmail.com>
The addition of inotify can introduce scenarios where file transfers complete abruptly, resulting in a corrupt/incomplete file.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
The logic used to sync requests to modify contents in hidden directories is no longer necessary, thus removed.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
The metadata won't always depict an actual change to the content of a file. Compare a hash of the file content rather than retrieve modified times to determine if it is necessary to install dependencies.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Re-entrant calls to "run" and "run_with_response" would poison the process state resulting in unexpected behavior. Use a lock to prevent this.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Reset the command tracking data before each retry. If the command is cancelled by the user immediately break.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Attempt to recover from "loose object" error in git status and git fetch commands. It is no longer necessary to run git fsck on every refresh attempt, only run it after a recovery attempt.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
When no callback is set for stderr, the outputfrom stderr is piped to stdout. This makes setting the callback redundant.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Image files (.png) are still granted unauthorized access, however all other files require that the request be authorized.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
While inotify watches are not applied to hidden directories it is still valid to use Moonraker's endpoints to perform operations on them. When performing an operation on an item within a hidden directory, or on a hidden directory itself, do not sync the request with inotify.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This allows clients to display a list of warnings that Moonraker detects. Previously this info was only logged.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Clients may pass a json web token via the query string's "access_token" argument to authorize requests that do not allow modified headers.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
When "force_logins" is enabled a user login is required if at least one user is registered, overriding the "trusted_clients" configuration.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Replace non-ASCII characters with "?" and add escaped unicode version of the file name to the "Content-Disposition" header to fix problems with thumbnails and downloading for the gcode file with non-ASCII characters in filename.
Signed-off-by: Uladzimir Palekh <visor.rti@gmail.com>
This root may be used to store log files for Klipper, Moonraker, and other applications as necessary.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
USER CHANGE: The deprecated "client_repo" and "client_path" options have been removed in this commit. Use [update_manger client client_name] to configure clients.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This includes a refactoring of the PowerDevice base class so that it defines some abstract methods that its children must implement.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Annotate function definitions, class attributes, and local variables as necessary for type hinting. This is useful for IDEs with linters that support type hints and also can be used by GitHub Actions to detect erroneous code.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Add a SentinelClass singleton. This can be used to generate Sentinels by
modules that need to use the Sentinel pattern for default values. This
allows for more robust type checking, as the Sentinel type should not be
included as a return value.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
On slower platforms it is possible for this to take time and block the event loop. Run shutil.rmtree in another thread to prevent this from occuring.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
If an inotify event is generated as the result of an API request, the notifications are synchronized so that they occur after the request returns. Use a mutex to prevent multiple "write" requests processing concurrently.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
If a file is moved from one location to another in the "gcodes" root it isn't necessary to rescan the file metadata as the underlying file information does not change. Add the ability to change the metadata's key and move associated thumbnails when a file is moved.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Since filelist_changed notifications are now delayed until after metadata is processed there is no need to send an additional metadata notification.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Rather than tracking watch paths directly, wrap each watch in a node. This makes makes it a bit easier to handle move and delete changes, as we don't have to manage several different dicts to store the information we need.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
When a directory is created, attempt to suppress notifications generated as its children are created.
Wait until all items are copied before notifying clients and scanning metadata.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
The copy methods are blocking and will block the asyncio event loop. Run them in a ThreadPoolExecutor to keep the event loop free.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
When a directory is deleted this attempts to suppress delete events for all children. Clients only need to be notified of the parent node, as its deletion implies all children have been deleted.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This guarantees that .ufps will not be unzipped in parallel and offloads a potential blocking operation to another process.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This removes all calls to "notify_filelist_changed" in the FileManager class. This also simplfies the previous `_convert_path()` method, as it is no longer necessary to return the path relative to the root.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Generally speaking static files do not require authorization, so there should be little harm in bypassing auth checks. This allows clients to retrieve assets without XHRs or requesting tokens.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
On some devices gpiod does not report the correct state. Since Moonraker maintains exclusive control over GPIO lines, its unnecessary to query the line itself for state, as we can maintain it ourselves.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This should allow git's internal timeout mechanisms to trigger at the expense of potentially delaying response time if a fetch or pull hangs.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
It is now possible for any authorized request to delete a user, however a logged in user cannot delete its own account.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
A single endpoint that can be used to both toggle and retrieve device state is useful for interoperability with home automation software. The previous endpoints will remain as they are still useful for batch operations.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Make sure that the current device state is correct in the event that a device is toggled by an external source.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Always run git fsck when initializing a repo. If a loose object error is detected when running git pull or git fetch, attempt to remove loose objects and retry.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Don't continuously log a change in throttled state, only log when a new throttled flag is detected.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Log the current user for each request. Dont' log requests that return with 200, 204, 206, or 304 in release mode.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
A Moonraker component may now register a namespace as "forbidden". API requests have no read/write access to forbidden namespaces and this cannot be overridden by the database debug option.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
CONFIG CHANGE: This deprecates the "enabled" option in the [authorization] section. Authorization will be enabled if the section is included in moonraker.conf, otherwise it will be disabled.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Force an update to cached file hashes should the modified time change. Re-enable the "should_return_304" check so clients can re-implement caching.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Some static files, such as logs, can change size during a request. This results in a content length mismatch and error. Cap the amount read based on the original content length.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Add a timer parameter to shelly and tasmota integration.
Use case: 3D printer mains power is connected to one of these devices. Just turning off the smart switch will just kill power to the RaspberryPI. Adding a timer will allow the PI to shutdown cleanly before killing the power
Signed-off-by: Dominik Weis fsironman@gmail.com
This allows protected namespaces to be modified externally. This is for dev use only, modifying protected namespaces can potentially corrupt the database.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This fixes a bug where it is protected namespace can be modified in an optional component if that component is disabled.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
add a check and handle consequently in case tasmota device is only one relay and would return a power value when asked a power1 value
Signed-off-by: Arnaud Schaer <arnaud.schaer@wanadoo.fr>
The Klipper repo always initializes last. Send a notification at this time, so if any clients are connected they can receive the full update status after init.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
During normal updates dependencies will always been updated when necessary. It is possible to force a dependency updating using the new "/machine/update/recover" endpoint.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Gives callers flexibility as to which signal is used to cancel a command first. By default shell_command will start with SIGTERM, then proceed to SIGKILL.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Rather than attempt to automate recovery, provide an endpoint for clients
to initiate. Clients can choose between a "hard" recovery, which removes
a corrupt repo and restores from a backup, or a "soft" recover which attempts
a git clean followed by a git reset.
Backups are now only taken when a valid repo is detected.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Note that this does not replace references to "Cura" or "Octoprint" plugins, which are also included in this module.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
It isn't necessary for each updater to have their own init lock. Combine them all into a single Event that is set after the "initialize_updaters" method completes.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Give more time for a standard command to complete. This may help prevent a git command from being terminated prematurely.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Git repos can occasionally become corrupted when attempting to deploy updates thought automated means. Use rysnc to create a backup of each git repo, allowing the repo to be restored in the event of a corrupted repo.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Since "run_with_response" now raises an exception there is no need to inspect the response for a None type. Instead the exception can be handled by the caller directly.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This error contains the return code and anything received from stdout and stderr when "run_with_response" encounters an error. This allows the caller to receive the output and easily distinguish between a successfully run command and an erronous command.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Sometimes git will erroneously leave the lock file in place when a command fails. Check for locked git repo before initialization, waiting up to 60 seconds before forcefully removing the lock. Also check for a locked repo after a failed fetch or pull, removing it if it exists.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Git clients may specify a branch other than "master" as the primary to pass the validation checks.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
This field does not persist in the database as it is subject change if a file is moved or deleted when Moonraker is not running.
Also add thumbnail information to the metadata sans the base64 image data for each entry.
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
Eric Callahan