Commit Graph

1203 Commits

Author SHA1 Message Date
Eric Callahan 3edb4b1bad tests: add support for loading a database
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan f2f1d413d4 scripts: introduce dbtool
This tool may be used to backup and restore Moonraker's lmdb
database without depending on the "lmdb-utils" package.  The
backup is done to a plain text file in cdb format, so a backup
may be restored on any platform.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 55baad1ad0 tests: rework path_args and klippy fixtures
Initialize the temporary directory in a "session_args" fixture that is session
scoped.  Move the Klippy process to a session scope, restarting the process
only when necessary.  This combination speeds up tests as it reduces disk
I/O and the overhead of starting the Klippy process for each test.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 81f2418d78 tests: fix error in websocket_client fixture
A jsonrpc request is a notification when no ID is present.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 348d45e87c tests: increase the timeout to 60 seconds
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan e6b4e54bea tests: fix the server pending tasks tests
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 6946f67dbf tests: increase the klippy process timeout to 2.5 seconds
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan f7f487ac18 database: sort protected namespaces before storing them
This makes the order in which they are reported consistent,
as the namespaces are stored in a set.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 888fb58c4f database: sync namespace bugfix
Use the transaction object to overwrite keys so that we don't
move the cursor.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 2b7a5ff035 tests: assign the WebsocketError to an static variable
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan d630b79e8b tests: http client fixes
Correctly encode the query string.  Use the query string for DELETE
requests in addition to GET requests.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 5f96822c90 database: don't overwrite "get" server errors
Make sure the correct exception is raised when "get_item" fails.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 2d28a3eeff database: fix typo in http exception
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 96b00549cf database: validate that namespace exists in ns_contains
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 0bd12c8dec database: don't allow sync against an empty dict
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan f3cd177a02 database: move_batch() fix
Raise an exception if the key lengths for the source and destination
keys do not match.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 11e299558d database: fix insert_batch()
If a namespace does not exist insert batch must create a new one rather
than raise an exception.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan bb39a3c1b2 database: handle invalid reduce results
It is possible to reduce to a value that is not a dict in "insert_item()"
and "update_item()".  Raise a ServerError with a clear description of
what went wrong in these situations.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 89e78b9a00 database: fix type reported in overwrite warning
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan c1104617b6 database: fix typo in sync implementation
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 3c92f42d49 database: allow synchronous calls before the server starts
It is possible that Moonraker can be instantiated while the eventloop
is running during tests.  To avoid errors allow synchronous calls until
the server starts.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:38 -05:00
Eric Callahan 9ab6d0e83d update_manager: support database changes
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-22 07:47:34 -05:00
Eric Callahan c081fa49a1 history: update for database changes
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan 9e57db0611 file_manager: add method to retreive the metadata storage object
This allows other components to synchronously access and update
metadata if necessary.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan 0dd10ce116 file_manager: update for changes in the database
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan b43f4623fc authorization: update for changes in the database
Since the User DB is not going to be large cache the users
in local memory and sync with the DB when changes are
made to the local user store.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan 46f74329d3 database: reduce duplicate code
Add a _get_db() method that perform the check for existance.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan e2a62f80d4 database: introduce record based operations
Now that all transactions run in a thread it is possible for them to
interleave.  The record based operations allow for batch operations
within a single transaction.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan e029b6c582 database: refactor to remove duplicate code
Wrap command implementations in with a _run_command() method.  All
database commands now return a Future object.  If the command was
run before the eventloop starts its possible to immediately query
the Future's result.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan 65d1f23352 database: add a sync method
Similar to the update method, however sync will remove any
keys in the database not in the new value.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan 781e3e6250 database: combine namespace wrappers
Rather than creating two wrappers, use a single wrapper whose methods
always return a future or awaitable.  If the operation occurs during
the __init__() method of a component it will be syncrhonous, and the
result from the future can be immediately queried.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan 4d0a43cb25 database: refactor initial check for empty keys
Rather than reading out the entire database, attempt to pop off
empty keys in each database.  This should should speed up
init on platforms with slow disk I/O and prevents reading
large namespaces in memory.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan e754d123dd database: improve ns_contains method
It isn't necessary to read out the entire namespace, just attempt
to fetch the record and reduce the key list.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan 43c1b998b0 database: add methods to clear and update a namespace
Since transactions are now threaded we need updating
and clearing a namespace needs to be performed within
one transaction.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan e895b10ac3 database: run transactions in a thread after the eventloop starts
Database transactions can perform disk I/O tasks, which at times will
block the eventloop.  Run all transactions in threads to prevent this
from occuring.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan 1be639b99a eventloop: use the default executor for run_in_thread()
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan ac55b95c1d database: Add unsafe shutdowns to the log rollover
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:57 -05:00
Eric Callahan 5f9706f6be test: initial testing framework
This adds the framework for unit testing Moonraker via pytest.
Initally only moonraker.py, klippy_connection.py, and confighelper.py
have acceptable coverage.  Coverage for other modules will be added on
an incremental basis, when most of Moonraker's source is covered tests
will be conducted via GitHub actions.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-17 11:19:41 -05:00
Lars Pöpperl 34f607c061
docs: add alternative sample file for home assistant
Signed-off-by: Lars Pöpperl  <lars.poepperl@gmail.com>
2022-02-17 11:03:30 -05:00
Simon Germain 11f4ae75cc
octoprint_compat: add webcam settings
Signed-off-by: Simon Germain <sgermain06@gmail.com>
2022-02-17 10:46:05 -05:00
Eric Callahan f79439aedd
funding: add a ko-fi button
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-16 05:44:10 -05:00
Eric Callahan c2861a1391 app: correct location header encoding
SIgned-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-15 14:39:33 -05:00
Eric Callahan e2d92b5575 confighelper: use read_file() rather than read()
The read() method will return an empty list if the file specified
does not exist or if the read failed in some fashion.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-15 11:42:50 -05:00
Eric Callahan 0e27a949af docs: update power documentation for "smartthings" devices
Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-14 06:26:34 -05:00
Eric Callahan 10703321b0 power: fetch the Smart Things token option as a template
This allows users to store their token in a "secrets" file

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-14 06:19:37 -05:00
Mitch Gallman 0dc05eac76 docs: add smartthings power documentation
docs: add smartthings power documentation

Signed-off-by: Mitch Gallman <mitchell.gallman@gmail.com>
2022-02-14 06:12:52 -05:00
Mitch Gallman d01b8a9763 power: add support for smartthings switch
power: add support for smartthings switch

Signed-off-by: Mitch Gallman <mitchell.gallman@gmail.com>
2022-02-14 06:12:52 -05:00
Eric Callahan b60153c919 workflows: disable blank issues
Signed-off-by: Eric Callahan <arksine.code@gmail.com>
2022-02-13 20:50:22 -05:00
Eric Callahan ee11ed2a15 confighelper: fix the get_hash() method
Include the option's values in the hash.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-13 20:48:27 -05:00
Eric Callahan 9be93ff445 docs: add "info_tags" to update manager status
This also fixes some stale documentation.

Signed-off-by:  Eric Callahan <arksine.code@gmail.com>
2022-02-13 11:32:32 -05:00