diff --git a/README.md b/README.md index fb5c5cb7f..252727152 100644 --- a/README.md +++ b/README.md @@ -1,96 +1,28 @@ [Steampipe Logo](https://steampipe.io?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) -[![plugins](https://img.shields.io/badge/apis_supported-140-blue)](https://hub.steampipe.io/plugins?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme)   -[![benchmarks](https://img.shields.io/badge/controls-5965-blue)](https://hub.steampipe.io/mods?objectives=compliance?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme)   -[![dashboards](https://img.shields.io/badge/dashboards-738-blue)](https://hub.steampipe.io/mods?objectives=dashboard?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme)   +[![plugins](https://img.shields.io/badge/apis_supported-140-blue)](https://hub.powerpipe.io/plugins?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme)   [![slack](https://img.shields.io/badge/slack-2297-blue)](https://turbot.com/community/join?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme)   [![maintained by](https://img.shields.io/badge/maintained%20by-Turbot-blue)](https://turbot.com?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) Steampipe is the zero-ETL solution for getting data directly from APIs and services. We offer these Steampipe engines: -- [Steampipe CLI](#steampipe-cli). Query APIs, check compliance, visualize on dashboards. +- Steampipe CLI. Use the Steampipe core engine to translate APIs to tables in the Postgres instance that's bundled with Steampipe. -- [Steampipe Postgres FDWs](https://steampipe.io/docs/steampipe_postgres/index). Use native Postgres Foreign Data Wrappers that translate APIs to foreign tables. +- Steampipe Postgres FDWs. Use [native Postgres Foreign Data Wrappers](https://steampipe.io/docs/steampipe_postgres/overview) to translate APIs to foreign tables. -- [Steampipe SQLite extensions](https://steampipe.io/docs/steampipe_sqlite/index). Use SQLite extensions that translate APIS to SQLite virtual tables. +- Steampipe SQLite extensions. Use [SQLite extensions](https://steampipe.io/docs/steampipe_sqlite/overview) to translate APIS to SQLite virtual tables. -- [Steampipe export tools](https://steampipe.io/docs/steampipe_export/index). Use standalone binaries that export data from APIs, no database required. +- Steampipe export tools. Use [standalone binaries](https://steampipe.io/docs/steampipe_export/overview) that export data from APIs, no database required. -- [Turbot Pipes](https://turbot.com/pipes). Query, check, and visualize with your team using the only intelligence, automation & security platform built specifically for DevOps. +- Turbot Pipes. Use [Turbot Pipes](https://turbot.com/pipes) to run Steampipe in the cloud. -## A common suite of API plugins +## Steampipe API plugins -The Steampipe community has grown a suite of [plugins](https://hub.steampipe.io/plugins?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) that map APIs to tables. They work with all Steampipe engines. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
CloudAWS, Alibaba, Azure, GCP, IBM, Oracle …
SaaSAirtable, Jira, GitHub, Google Workspace, Microsoft 365, Salesforce, Slack, Stripe, Zoom …
SecurityCrowdStrike, GitGuardian, Have I Been Pwned, PAN-OS, Shodan, Trivy, VirusTotal …
IdentityAzure AD, Duo, Keycloak, Google Directory, LDAP …
DevOpsDocker, Grafana, Kubernetes, Prometheus …
NetBaleen, Cloudflare, crt.sh, Gandi, IMAP, ipstack, Tailscale, updown.io, WHOIS …
IaCCloudFormation, Terraform …
LogsAlgolia, AWS CloudWatch, Datadog, Splunk …
SocialHackerNews, Twitter, Reddit, RSS …
Your APIBuild your own custom plugins
- - -## Steampipe CLI - -With [Steampipe CLI](https://steampipe.io?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) you can: - -- **Query** → Use SQL to [query](https://steampipe.io/docs/query/overview?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) (and join across!) [APIs](https://hub.steampipe.io/plugins?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme). - -- **Check** → Ensure that cloud resources comply with [security benchmarks](https://steampipe.io/docs/check/overview?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) such as CIS, NIST, and SOC2. - -- **Visualize** → View [prebuilt dashboards](https://steampipe.io/docs/dashboard/overview?objectives=dashboard&utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) or [build your own](https://steampipe.io/docs/mods/writing-dashboards?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme). - +The Steampipe community has grown a suite of [plugins](https://hub.powerpipe.io/plugins?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) that map APIs to database tables. They work with all Steampipe engines. -The [interactive query shell](https://steampipe.io/docs/query/query-shell?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) is one way you can query those tables. - - - -You can also use psql, pgcli, Metabase, Tableau, or [any client](https://steampipe.io/docs/cloud/integrations/overview?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) that can connect to Postgres. - -### Get started with the CLI - -
- - Install Steampipe -
+## Install Steampipe The downloads page shows you how but tl;dr: @@ -107,70 +39,39 @@ brew tap turbot/tap brew install steampipe ``` -
+## Install a plugin -
- Add a plugin -
- - Choose a plugin from the [hub](https://hub.steampipe.io?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme), for example: [Net](https://hub.steampipe.io/plugins/turbot/net?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme). + Choose a plugin from the [hub](https://hub.powerpipe.io?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme), for example: [Hacker News](https://hub.powerpipe.io/plugins/turbot/hackernews?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme). Run the `steampipe plugin` command to install it. ```sh -steampipe plugin install net -``` -
- -
- Run steampipe query -
-Launch the interactive shell. - -```sh -steampipe query +steampipe plugin install hackernews ``` -Run your first query! +## Query with a plugin + +Run a query using `psql` — or another Postgres client , or [Powerpipe](https://powerpipe.io) — to query a table provided by the installed plugin. + -```sql -select - * -from - net_certificate -where - domain = 'google.com'; ``` -
+psql -h localhost -p 9193 -d steampipe -U steampipe +``` -
- Learn more about the CLI +``` +steampipe=> select * from hackernews_new limit 10 +``` -- It's [just SQL](https://steampipe.io/docs/sql/steampipe-sql?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme)! - -- You can run queries [on the command line](https://steampipe.io/docs/query/overview?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme#non-interactive-batch-query-mode) and include them in scripts. - -- Other [commands](https://steampipe.io/docs/reference/cli/overview?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) run benchmarks, launch Steampipe as a service, and start the dashboard server. - -- [Meta-commands](https://steampipe.io/docs/reference/dot-commands/overview?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) control caching, [environment variables](https://steampipe.io/docs/reference/env-vars/overview), the [search path](https://steampipe.io/docs/guides/search-path?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme), and more. - -- Queries can run in [batch mode](https://steampipe.io/docs/query/batch-query?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme). - -- You can bundle connections (e.g. for many AWS accounts) using an [aggregator](https://steampipe.io/docs/managing/connections?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme#using-aggregators). - -
- -
- Build and develop the CLI +## Developing Prerequisites: - [Golang](https://golang.org/doc/install) Version 1.19 or higher. -Clone: +Clone the repo. ```sh -git clone git@github.com:turbot/steampipe +git clone https://github.com/turbot/steampipe cd steampipe ``` @@ -180,176 +81,28 @@ Build, which automatically installs the new version to your `/usr/local/bin/stea make ``` -Check the verison +Check the version. ``` $ steampipe -v -steampipe version 0.18.1 +steampipe version 0.21.1 ``` -Install a plugin +Install a plugin and run a query as per above. -``` -$ steampipe plugin install steampipe -``` - -Try it! - -``` -steampipe query -> .inspect steampipe -+-----------------------------------+-----------------------------------+ -| TABLE | DESCRIPTION | -+-----------------------------------+-----------------------------------+ -| steampipe_registry_plugin | Steampipe Registry Plugins | -| steampipe_registry_plugin_version | Steampipe Registry Plugin Version | -+-----------------------------------+-----------------------------------+ - -> select * from steampipe_registry_plugin; -``` -
- -### Steampipe Mods: Dashboards and benchmarks - -The Steampipe community has also grown a suite of [mods](https://hub.steampipe.io/mods?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) which are sets of **dashboards** that visualize your resources and **benchmarks** that check your cloud resources for compliance. - - - - - - - - - - - - - - - - - - - - - - - - - -
ComplianceCheck AWS, Azure, GCP, etc for compliance with HIPAA, PCI, etc -
CostReview what AWS, Azure, GCP, and other clouds are costing you
InsightsVisualize cloud resources with charts, tables, and interactive widgets
SecurityUse CIS, NIST, FedRAMP etc to assess the security of AWS, Azure, GCP, etc
TagsVerify the consistency of tags applied to AWS, Azure, and GCP resources
Your modBuild your own benchmarks and dashboards
- - - -### Running dashboards and benchmarks - -![benchmarks-and-dashboards](https://user-images.githubusercontent.com/46509/193875366-7d10ca8b-601a-4d93-a333-5c62ea86374b.gif) - -Dashboards and benchmarks use SQL to gather data and HCL to flow the data into [dashboard widgets](https://steampipe.io/blog/dashboards-as-code?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme) and [benchmark controls](https://steampipe.io/blog/release-0-11-0?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme#composable-mods). You can use the existing suites of benchmarks and dashboards, or build derivative versions, or create your own. - -### Get started with dashboards and benchmarks - -
-Install the Net Insights mod -
-The Net Insights mod works with the Net plugin shown above. To run it, first clone its repo and change to that directory. - -```sh -git clone https://github.com/turbot/steampipe-mod-net-insights -cd steampipe-mod-net-insights -``` -
- -
-
-Run benchmarks in the CLI - -All the benchmarks: - -```sh -steampipe check all -``` - -A single benchmark: - -```sh -steampipe check benchmark.dns_best_practices -``` - -A single control: - -```sh -steampipe check control.dns_ns_name_valid -``` - -Available formats include JSON, CSV, HTML, and ASFF. -You can use custom output templates to create new output formats. -
- -
-Run benchmarks as dashboards -
-Launch the dashboard server: `steampipe dashboard`, then open `http://localhost:9194` in your browser. - -The home page lists available dashboards. Click `DNS Best Practices` to view that dashboard. - -Note that the default domains are `microsoft.com` and `github.com`. You can change those defaults to check other domains. -
- -
-Use dashboards to explore your resources -
-Dashboards use charts, tables, and interactive widgets to help you explore and visualize your resources. - -The AWS Insights mod, for example, provides dozens of dashboards that exercise the full set of widgets. To use these dashboards, first install the AWS plugin and authenticate. Then clone `AWS Insights`, change to its directory, launch `steampipe dashboard`, and open `localhost:9194`. -
- -## Open Source & Contributing +## Open source & contributing This repository is published under the [AGPL 3.0](https://www.gnu.org/licenses/agpl-3.0.html) license. Please see our [code of conduct](https://github.com/turbot/.github/blob/main/CODE_OF_CONDUCT.md). Contributors must sign our [Contributor License Agreement](https://turbot.com/open-source#cla) as part of their first pull request. We look forward to collaborating with you! [Steampipe](https://steampipe.io) is a product produced from this open source software, exclusively by [Turbot HQ, Inc](https://turbot.com). It is distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our [Open Source FAQ](https://turbot.com/open-source). -## Get Involved - -**[Join #steampipe on Slack →](https://turbot.com/community/join)** - -Want to help but don't know where to start? Pick up one of the `help wanted` issues: -* [Steampipe](https://github.com/turbot/steampipe/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22) - ## Turbot Pipes Want a hosted version of Steampipe? Bring your team to [Turbot Pipes](https://pipes.turbot.com?utm_id=gspreadme&utm_source=github&utm_medium=repo&utm_campaign=github&utm_content=readme). -## License +## Get involved -Steampipe is distributed under [AGPL-3.0](https://github.com/turbot/steampipe/blob/main/LICENSE). +**[Join #steampipe on Slack →](https://turbot.com/community/join)** +Want to help but don't know where to start? Pick up one of the `help wanted` issues: +* [Steampipe](https://github.com/turbot/steampipe/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22) \ No newline at end of file