59 Commits

Author SHA1 Message Date
Puskar Basu
3f4eaae1a8 Fix db client deadlocks with non-blocking cleanup and RW locks (#4918) 2025-12-16 21:19:27 +05:30
Nathan Wallace
55e2d407a0 Fix #4783: Nil pointer dereference in updateConnectionSchema (#4901)
* Add test demonstrating bug #4783 - updateConnectionSchema with nil pool

This test verifies that updateConnectionSchema handles a nil pool gracefully.
While RefreshConnections (via newRefreshConnectionState) already checks for
nil pool since #4778, this test demonstrates that updateConnectionSchema
should perform an early nil check for better error handling.

The test currently passes because newRefreshConnectionState catches the nil
pool, but we should add an explicit check at the start of updateConnectionSchema
for clarity and to avoid unnecessary work.

Related to issue #4783

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix #4783: Add nil pool check in updateConnectionSchema

Add an early nil check for the pool at the beginning of updateConnectionSchema
to prevent unnecessary work and provide clearer error handling.

While newRefreshConnectionState (called by RefreshConnections) already checks
for nil pool since #4778, adding the check at the start of updateConnectionSchema
provides several benefits:

1. Avoids unnecessary work - we don't call RefreshConnections if pool is nil
2. Clearer error logging - warning message specifically indicates the issue
   is in updateConnectionSchema
3. Defense in depth - validates preconditions before executing the method

The method is called from the message server when a plugin sends a schema
update notification, so the nil check ensures we handle edge cases gracefully.

Closes #4783

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-17 04:00:18 -05:00
Nathan Wallace
754c7e6832 Nil pointer dereference in waitForPluginLoad closes #4752 (#4896)
* Add test for #4752: Nil pointer dereference in waitForPluginLoad

This test demonstrates the bug where waitForPluginLoad() panics with a nil
pointer dereference when a plugin fails during startup before the reattach
config is set.

The test creates a runningPlugin with reattach=nil and closes the failed
channel, simulating a plugin that fails in startPluginProcess before
initializePlugin is called.

* Fix #4752: Add nil check for reattach in waitForPluginLoad

Adds a nil pointer check before accessing p.reattach.Pid when a plugin
fails during startup. If the plugin fails before the reattach config is
set (e.g., in startPluginProcess), reattach will be nil and the code
would previously panic when trying to log the PID.

The fix checks if reattach is nil and logs an appropriate message in
each case, preventing the panic while still providing useful debugging
information.
2025-11-16 15:53:59 -05:00
Nathan Wallace
96d346ae5b HandlePluginLimiterChanges nil pool handling closes #4785 (#4877)
* Add test demonstrating bug #4785 - HandlePluginLimiterChanges panics with nil pool

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com)

* Fix #4785: Add nil pool check in refreshRateLimiterTable

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-16 11:09:57 -05:00
Nathan Wallace
9c3791fafd Fix #4809: Add nil checks for sessions map after Close() (rebased) (#4883)
* Add test for #4809: BeforeClose should handle nil sessions map

This test verifies that the BeforeClose callback checks if c.sessions map
has been nil'd by Close() before attempting to delete from it.

The test fails as expected without the fix, proving the bug exists.

Bug: #4809

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix #4809: Add nil check in BeforeClose for sessions map

Add nil check in BeforeClose callback before accessing c.sessions map.
This prevents panic when the callback executes after Close() has nil'd
the map.

Fixes #4809

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-16 10:53:39 -05:00
Nathan Wallace
e02d7d7d0d Nil pointer check in PluginManager.Shutdown closes #4782 (#4823)
* Add test demonstrating bug #4782 - Shutdown panics with nil pool

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix #4782: Add nil pool check in PluginManager.Shutdown

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-15 20:17:26 -05:00
Nathan Wallace
58b41541e5 Fix flaky concurrent rate limiter tests (#4892)
Fixes two flaky tests that were failing intermittently in CI:
- TestPluginManager_ConcurrentUpdateRateLimiterStatus
- TestPluginManager_ConcurrentRateLimiterMapAccess2

Root cause: The test writer goroutines were modifying pm.userLimiters
without mutex protection, while reader goroutines were simultaneously
accessing the map. Go's map implementation detects concurrent read/write
access and panics with "fatal error: concurrent map read and map write".

The production code in handleUserLimiterChanges (lines 98-100) correctly
protects writes with pm.mut.Lock/Unlock. The tests needed to simulate
this same behavior.

Changes:
- Added missing mut field initialization in TestPluginManager_ConcurrentUpdateRateLimiterStatus
- Wrapped all pm.userLimiters writes in both tests with pm.mut.Lock/Unlock
- Added comments explaining the mutex usage matches production code

Testing: Ran tests 100+ times consecutively with and without -race flag,
all passed successfully. Previously failed on 2nd iteration without the fix.
2025-11-15 19:52:06 -05:00
Nathan Wallace
4f1367fe6c Race condition in updateRateLimiterStatus closes #4786 (#4830)
* Add test demonstrating bug #4786 - race in updateRateLimiterStatus

Test demonstrates the race condition when updateRateLimiterStatus reads
from the userLimiters map without holding a lock, while another goroutine
concurrently writes to the map.

Running with -race flag shows data races at:
- plugin_manager_rate_limiters.go:176 (read in getUserDefinedLimitersForPlugin)
- plugin_manager_rate_limiters.go:161 (read in updateRateLimiterStatus)
- rate_limiters_test.go:40 (concurrent write)

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix #4786: Protect updateRateLimiterStatus with RWMutex

Add RWMutex write lock protection to updateRateLimiterStatus to prevent
race conditions when the method reads from userLimiters map and writes to
pluginLimiter.Status fields while other goroutines concurrently modify these
data structures.

The fix uses m.mut.Lock() (not RLock) because the method modifies the
pluginLimiter.Status field, requiring exclusive write access.

Note: This fix assumes updateRateLimiterStatus is not called from within
a context that already holds the mutex. If it is, additional refactoring
will be needed to prevent deadlock.

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix deadlock in updateRateLimiterStatus

The previous fix introduced a deadlock where updateRateLimiterStatus() would:
1. Acquire m.mut.Lock()
2. Call getUserDefinedLimitersForPlugin()
3. Which tries to acquire m.mut.RLock() - deadlock!

Go mutexes cannot acquire RLock when the same goroutine already holds Lock.

Fix by refactoring into internal/public pattern:
- getUserDefinedLimitersForPlugin() - public, acquires RLock
- getUserDefinedLimitersForPluginInternal() - internal, no lock (caller must hold it)
- updateRateLimiterStatus() now calls internal version while holding lock

Test verification: TestPluginManager_UpdateRateLimiterStatus_NoOverride
- Before: Timeout after 28s
- After: Pass in 0.429s

Fixes #4786

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-15 11:47:09 -05:00
Nathan Wallace
d58888e2d2 Nil pointer dereference in OnConnectionConfigChanged closes #4784 (#4829)
* Add test demonstrating bug #4784 - OnConnectionConfigChanged panics with nil pool

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix #4784: Add nil pool check in handlePluginInstanceChanges

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-15 11:32:47 -05:00
Nathan Wallace
6016a71053 Nil pointer dereference in logReceiveError closes #4798 (#4842)
* Unskip test demonstrating bug #4798: Nil pointer dereference in logReceiveError

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix #4798: Add nil check to logReceiveError

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-15 10:50:17 -05:00
Nathan Wallace
7a1533bd4f Race condition in rate limiter map access closes #4799 (#4843)
* Add test for #4799: Race condition in rate limiter map access

This test demonstrates the race condition in getUserDefinedLimitersForPlugin
where the userLimiters map is read without mutex protection while being
concurrently written by other goroutines.

Run with: go test -race -v -run TestPluginManager_ConcurrentRateLimiterMapAccess ./pkg/pluginmanager_service

* Fix #4799: Add mutex protection for rate limiter map access

Protected all accesses to m.userLimiters map with RWMutex:
- getUserDefinedLimitersForPlugin: Added RLock for map read
- getPluginsWithChangedLimiters: Added RLock for map iteration
- handleUserLimiterChanges: Added Lock for map write
- refreshRateLimiterTable: Added RLock for map iteration
- setRateLimiters: Added RLock for map read

This prevents data races when the map is concurrently read and written
by multiple goroutines.
2025-11-15 10:48:09 -05:00
Nathan Wallace
2e5f3fda97 Add comprehensive passing tests from bug hunting initiative (#4864) 2025-11-13 09:26:46 +08:00
Puskar Basu
e19d35c457 chore: update module to v2 and bump Go version to 1.24 (#4597) 2025-07-07 16:03:56 +05:30
Puskar Basu
9927cefece Fix issue where plugin start timeout was getting limited to 60s. Closes #4477 (#4493) 2025-03-31 13:00:22 +01:00
Puskar Basu
96959081b0 Added RateLimiterFromProto and RateLimiterasProto (#4500) 2025-03-12 22:14:14 +05:30
Puskar Basu
da2f3ecc03 Upgrade to pipe-fittings v2, go-kit v1 (#4485) 2025-03-06 16:34:18 +05:30
kai
112647bae0 tidy 2024-10-15 11:44:13 +01:00
kai
84fc15b350 rebase fixes 2024-09-27 18:14:28 +05:30
kai
cb681c67cc compiles but needs testing 2024-09-27 18:14:28 +05:30
kai
80ad514e1d steampipe compiles 2024-09-27 18:14:28 +05:30
kai
fd94b2e2ec working on it 2024-09-27 18:14:25 +05:30
kai
cd07bf20f1 working on it 2024-09-27 18:13:12 +05:30
kai
faead86d26 move stuff to pipe-fittings 2024-09-27 18:13:12 +05:30
Patrick Decat
d3adb67f6d Fix: Update plugin manager to properly flag shutdown is happening. Closes #4365 2024-09-02 09:56:31 +01:00
Nathan Mische
8ba2cc5645 Make plugin start timeout configurable. (#4321) 2024-08-30 17:15:41 +05:30
kaidaguerre
e6e9714e4c Update all ErrorAndWarnings function returns to pass by value, removing possibility of nil ErrorAndWarnings. Closes #3974 (#4212) 2024-03-21 11:46:10 +00:00
kaidaguerre
43540d318c Add support for nested dashboards. Closes #4208 2024-03-20 15:58:57 +00:00
kaidaguerre
914c2866fb Remove need to generate plugin file hash when creating GRPC plugin client. Closes #4200 2024-03-15 11:39:45 +00:00
kaidaguerre
0db7768ce9 Improve startup performance with high plugin count - parallelize plugin startup. Closes #4183 2024-03-13 12:15:04 +00:00
kaidaguerre
d24c9d90db Updates for steampipe_plugin_column table. Closes #4022
* Fix inconsistent value for plugin in steampipe_plugin_column table - always use long name.
* Rename `plugin_name` column to `plugin`
* Ensure empty col values
2023-12-12 12:44:21 +00:00
kaidaguerre
46ef6ec90a Add steampipe_plugin_column introspection table to internal schema. Closes #4003
(cherry picked from commit 12079986d5)
2023-11-29 12:55:21 +00:00
kai
6407ff5982 Improve the warning output when plugins with incompatible SDK versions fail to load - update error message. #3911 2023-09-28 17:40:01 +01:00
kaidaguerre
2306767fd2 Improve the warning output when plugins with incompatible SDK versions fail to load. Closes #3911 2023-09-27 17:59:32 +01:00
kaidaguerre
b7710e31e5 re-add support for locally developed plugins in the folder plugins/local. Closes #3904 2023-09-26 16:07:35 +01:00
Puskar Basu
2438a122a6 Fixes issue where initialising rate limiter definitions was taking too long due to force recreation of tables. Closes #3902 2023-09-26 14:48:59 +01:00
kaidaguerre
49d58d4f52 Fix aggregator error no connection data loaded for child connection if more than one plugin instance is in use. Closes #3899 (#3898)
* Update PopulateChildren.PopulateChildren to take plugin instance into account
* Fix GetNewConnectionStateFromConnectionInsertSql to use connection names not map of connections
2023-09-25 17:14:01 +01:00
kaidaguerre
235cbc2037 Connection watching fixes.
Fix ConnectionConfigMap.Diff  inconsistent usage of Plugin and PluginInstance. Closes #3895
Fix GetConnectionStateErrorSql returning invalid query with additional parameter. Closes #3896
Update FDW to v1.8.0-rc.11 - fix failure to load connection config when importing schema
2023-09-25 12:35:00 +01:00
kaidaguerre
6f051ea463 Do not add connections in error to plugin manager connection map. Closes #3893 2023-09-22 18:12:29 +01:00
kaidaguerre
79606c5bcd Rename internal introspection tables. Fix warning notifications from RefreshConnections. Improve error handlingh for config inconsistencies in conneciton and plugin config. Closes #3886 2023-09-22 16:02:41 +01:00
kaidaguerre
f232b14f06 Add support for multiple instances of the same plugin version to execute. Closes #3863 2023-09-18 15:39:17 +01:00
Binaek Sarkar
acc3f942ea Installs only FDW when fdw files are not installed. Closes #2040 2023-09-18 12:23:56 +01:00
kaidaguerre
7feb305fde Provide mechanism for plugin manager to send warnings back to CLI. Closes #3603 (#3835) 2023-09-13 15:34:15 +01:00
kaidaguerre
95fed2ed2a Key the rate limiter and plugin config maps by plugin image ref, not short name. Closes #3820 2023-09-11 15:56:35 +01:00
Binaek Sarkar
862fcf1b71 Fix stall in plugin manager shutdown. Closes #3817 2023-09-08 17:48:20 +01:00
kaidaguerre
01f2f1e525 Update PluginManager plugins property after a ConnectionConfigChanged event. Fixes failure to start plugin after connection config added while service is running. Closes #3812 (#3813) 2023-09-07 18:48:07 +01:00
kaidaguerre
f041597497 Add support for plugin connection config and options blocks. Add options and env var configuration of max memory. Closes #3807 2023-09-06 15:59:13 +01:00
kaidaguerre
2a86d08445 Add support to retrieve plugin rate limiter definitions and use to populate steampipe_rate_limiter table. Closes #3805 (#3803) 2023-09-06 13:28:22 +01:00
Binaek Sarkar
08b447a261 Differentiate between user and system queries using application name. Closes #3600 2023-08-25 16:50:27 +01:00
kai
e5ac3eff06 Merge remote-tracking branch 'origin/main' 2023-08-11 15:30:15 +01:00
kaidaguerre
202cb68692 Add HCL support for defining rate limiters, with filewatching as per connection config. Closes #3746 2023-08-11 14:24:44 +01:00