Commit Graph

46 Commits

Author SHA1 Message Date
Puskar Basu
3f4eaae1a8 Fix db client deadlocks with non-blocking cleanup and RW locks (#4918) 2025-12-16 21:19:27 +05:30
Nathan Wallace
55e2d407a0 Fix #4783: Nil pointer dereference in updateConnectionSchema (#4901)
* Add test demonstrating bug #4783 - updateConnectionSchema with nil pool

This test verifies that updateConnectionSchema handles a nil pool gracefully.
While RefreshConnections (via newRefreshConnectionState) already checks for
nil pool since #4778, this test demonstrates that updateConnectionSchema
should perform an early nil check for better error handling.

The test currently passes because newRefreshConnectionState catches the nil
pool, but we should add an explicit check at the start of updateConnectionSchema
for clarity and to avoid unnecessary work.

Related to issue #4783

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix #4783: Add nil pool check in updateConnectionSchema

Add an early nil check for the pool at the beginning of updateConnectionSchema
to prevent unnecessary work and provide clearer error handling.

While newRefreshConnectionState (called by RefreshConnections) already checks
for nil pool since #4778, adding the check at the start of updateConnectionSchema
provides several benefits:

1. Avoids unnecessary work - we don't call RefreshConnections if pool is nil
2. Clearer error logging - warning message specifically indicates the issue
   is in updateConnectionSchema
3. Defense in depth - validates preconditions before executing the method

The method is called from the message server when a plugin sends a schema
update notification, so the nil check ensures we handle edge cases gracefully.

Closes #4783

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-17 04:00:18 -05:00
Nathan Wallace
754c7e6832 Nil pointer dereference in waitForPluginLoad closes #4752 (#4896)
* Add test for #4752: Nil pointer dereference in waitForPluginLoad

This test demonstrates the bug where waitForPluginLoad() panics with a nil
pointer dereference when a plugin fails during startup before the reattach
config is set.

The test creates a runningPlugin with reattach=nil and closes the failed
channel, simulating a plugin that fails in startPluginProcess before
initializePlugin is called.

* Fix #4752: Add nil check for reattach in waitForPluginLoad

Adds a nil pointer check before accessing p.reattach.Pid when a plugin
fails during startup. If the plugin fails before the reattach config is
set (e.g., in startPluginProcess), reattach will be nil and the code
would previously panic when trying to log the PID.

The fix checks if reattach is nil and logs an appropriate message in
each case, preventing the panic while still providing useful debugging
information.
2025-11-16 15:53:59 -05:00
Nathan Wallace
e02d7d7d0d Nil pointer check in PluginManager.Shutdown closes #4782 (#4823)
* Add test demonstrating bug #4782 - Shutdown panics with nil pool

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix #4782: Add nil pool check in PluginManager.Shutdown

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-15 20:17:26 -05:00
Nathan Wallace
7a1533bd4f Race condition in rate limiter map access closes #4799 (#4843)
* Add test for #4799: Race condition in rate limiter map access

This test demonstrates the race condition in getUserDefinedLimitersForPlugin
where the userLimiters map is read without mutex protection while being
concurrently written by other goroutines.

Run with: go test -race -v -run TestPluginManager_ConcurrentRateLimiterMapAccess ./pkg/pluginmanager_service

* Fix #4799: Add mutex protection for rate limiter map access

Protected all accesses to m.userLimiters map with RWMutex:
- getUserDefinedLimitersForPlugin: Added RLock for map read
- getPluginsWithChangedLimiters: Added RLock for map iteration
- handleUserLimiterChanges: Added Lock for map write
- refreshRateLimiterTable: Added RLock for map iteration
- setRateLimiters: Added RLock for map read

This prevents data races when the map is concurrently read and written
by multiple goroutines.
2025-11-15 10:48:09 -05:00
Puskar Basu
e19d35c457 chore: update module to v2 and bump Go version to 1.24 (#4597) 2025-07-07 16:03:56 +05:30
Puskar Basu
9927cefece Fix issue where plugin start timeout was getting limited to 60s. Closes #4477 (#4493) 2025-03-31 13:00:22 +01:00
Puskar Basu
96959081b0 Added RateLimiterFromProto and RateLimiterasProto (#4500) 2025-03-12 22:14:14 +05:30
Puskar Basu
da2f3ecc03 Upgrade to pipe-fittings v2, go-kit v1 (#4485) 2025-03-06 16:34:18 +05:30
kai
84fc15b350 rebase fixes 2024-09-27 18:14:28 +05:30
kai
cb681c67cc compiles but needs testing 2024-09-27 18:14:28 +05:30
kai
80ad514e1d steampipe compiles 2024-09-27 18:14:28 +05:30
kai
fd94b2e2ec working on it 2024-09-27 18:14:25 +05:30
kai
cd07bf20f1 working on it 2024-09-27 18:13:12 +05:30
kai
faead86d26 move stuff to pipe-fittings 2024-09-27 18:13:12 +05:30
Patrick Decat
d3adb67f6d Fix: Update plugin manager to properly flag shutdown is happening. Closes #4365 2024-09-02 09:56:31 +01:00
Nathan Mische
8ba2cc5645 Make plugin start timeout configurable. (#4321) 2024-08-30 17:15:41 +05:30
kaidaguerre
914c2866fb Remove need to generate plugin file hash when creating GRPC plugin client. Closes #4200 2024-03-15 11:39:45 +00:00
kaidaguerre
0db7768ce9 Improve startup performance with high plugin count - parallelize plugin startup. Closes #4183 2024-03-13 12:15:04 +00:00
kaidaguerre
12079986d5 Add steampipe_plugin_column introspection table to internal schema. Closes #4003 2023-11-29 12:49:12 +00:00
kai
6407ff5982 Improve the warning output when plugins with incompatible SDK versions fail to load - update error message. #3911 2023-09-28 17:40:01 +01:00
kaidaguerre
2306767fd2 Improve the warning output when plugins with incompatible SDK versions fail to load. Closes #3911 2023-09-27 17:59:32 +01:00
kaidaguerre
b7710e31e5 re-add support for locally developed plugins in the folder plugins/local. Closes #3904 2023-09-26 16:07:35 +01:00
kaidaguerre
49d58d4f52 Fix aggregator error no connection data loaded for child connection if more than one plugin instance is in use. Closes #3899 (#3898)
* Update PopulateChildren.PopulateChildren to take plugin instance into account
* Fix GetNewConnectionStateFromConnectionInsertSql to use connection names not map of connections
2023-09-25 17:14:01 +01:00
kaidaguerre
6f051ea463 Do not add connections in error to plugin manager connection map. Closes #3893 2023-09-22 18:12:29 +01:00
kaidaguerre
79606c5bcd Rename internal introspection tables. Fix warning notifications from RefreshConnections. Improve error handlingh for config inconsistencies in conneciton and plugin config. Closes #3886 2023-09-22 16:02:41 +01:00
kaidaguerre
f232b14f06 Add support for multiple instances of the same plugin version to execute. Closes #3863 2023-09-18 15:39:17 +01:00
Binaek Sarkar
acc3f942ea Installs only FDW when fdw files are not installed. Closes #2040 2023-09-18 12:23:56 +01:00
kaidaguerre
7feb305fde Provide mechanism for plugin manager to send warnings back to CLI. Closes #3603 (#3835) 2023-09-13 15:34:15 +01:00
kaidaguerre
95fed2ed2a Key the rate limiter and plugin config maps by plugin image ref, not short name. Closes #3820 2023-09-11 15:56:35 +01:00
Binaek Sarkar
862fcf1b71 Fix stall in plugin manager shutdown. Closes #3817 2023-09-08 17:48:20 +01:00
kaidaguerre
01f2f1e525 Update PluginManager plugins property after a ConnectionConfigChanged event. Fixes failure to start plugin after connection config added while service is running. Closes #3812 (#3813) 2023-09-07 18:48:07 +01:00
kaidaguerre
f041597497 Add support for plugin connection config and options blocks. Add options and env var configuration of max memory. Closes #3807 2023-09-06 15:59:13 +01:00
kaidaguerre
2a86d08445 Add support to retrieve plugin rate limiter definitions and use to populate steampipe_rate_limiter table. Closes #3805 (#3803) 2023-09-06 13:28:22 +01:00
Binaek Sarkar
08b447a261 Differentiate between user and system queries using application name. Closes #3600 2023-08-25 16:50:27 +01:00
kai
e5ac3eff06 Merge remote-tracking branch 'origin/main' 2023-08-11 15:30:15 +01:00
kaidaguerre
202cb68692 Add HCL support for defining rate limiters, with filewatching as per connection config. Closes #3746 2023-08-11 14:24:44 +01:00
kai
02654f8d4d Merge branch 'v0.20.x'
# Conflicts:
#	pkg/pluginmanager_service/plugin_manager.go
#	tests/acceptance/test_files/dynamic_schema.bats
2023-08-11 14:18:53 +01:00
Puskar Basu
c9dfc8df0b Fixes timing issue where refresh connections was sometimes not run when starting service. Closes #3734 2023-08-11 10:36:20 +01:00
kaidaguerre
d8a92f37bf When plugin startup experiences panic, plugin writes known string to stdout so plugin manager can intercept the "Unrecognized remote plugin message" and instead return the panic message. DO not retry plugin startup if initialisation fails. #3732. 2023-08-09 13:20:03 +01:00
kaidaguerre
43dd6c7a61 Refactor Plugin manager:remove support for plugins which do not support multiple connections, simplify startup.
If plugin process crashes, benchmark or dashboard runs can leave running plugin processes after shutdown. Fixes #3598
2023-06-21 16:18:49 +01:00
kai
fb19ee30ce Increase plugin load timeout. Closes #3564 2023-06-12 17:23:17 +01:00
kaidaguerre
e465a77412 Avoid orphan plugin processes when running short batch queries. Closes #3514 2023-06-01 17:07:32 +01:00
kaidaguerre
dcd0ced93a Update refresh connections to write comments for exemplar schemas first, followed by remaining schemas. Closes #3413 2023-05-16 14:13:02 +01:00
kaidaguerre
b0b2c62338 Add support for 'import_schema' property in connection config, controlling whether to create a postgres schema for a steampipe connection. Closes #3407
Rename 'connection_state' table to 'steampipe_connection_state'. #3403
Rename 'internal' table to 'steampipe_internal
Remove 'Connection' property from ConnectionState
2023-05-15 13:07:36 +01:00
kaidaguerre
40804a3201 Execute RefreshConnections asyncronously and optimise for high connection count. Add connection_state table.
- Execute RefreshConnections asyncronously
- Add connection_state table to indicate the loading state of connections
- Optimise RefreshConnections by cloning connection schemas
- Add locking to ensure only a single instance of RefreshConnections runs
- Start executing queries without waiting for connections to load, add smart error handling to wait for required connection
- Optimise autocomplete for high connection count
- Autocomplete and inspect data available before all conections are refreshed
- Update file watcher to respond to CHMOD, so thaat it pickes up deletion of file contents

 Closes #3394
 Closes #3267
2023-05-10 09:05:08 +01:00