2 Commits

Author	SHA1	Message	Date
Ben Church	fb7258e2bd	Move tools/ci_* projects to airbyte-ci, update to use Poetry, bump to python 3.10 (#27957) ... * Move ci_connector_ops * Move ci_credentials * Move tools/ci_common_utils * Rename tools to airbyte-ci * Move to ci * Convert ci_credentials * Convert ci_common_utls * Convert ci_connector_ops * Get pipelines running * Move pipelines to own poetry project * Update readme * Delete * Add ci_code_validator * Use pipx to install gha deps * Fix' * Ensure every thing is running * Automated Commit - Formatting Changes * Gitignore miss * Add pipx installer * Get local pipx dependencies * Fix paths * Install pipx * ceremonial source-faker change * Add installation step for ci_code_validator * Add comment * remove ci_code_validator * Address code review comments * add pipx install to acceptance-test-docker.sh * Run formater * Revert "ceremonial source-faker change" This reverts commit 26884cd0db. * gitignore lecacy pipeline report path * update poetry.lock * skip upload if logs do not exist --------- Co-authored-by: bnchrch <bnchrch@users.noreply.github.com> Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>	2023-07-26 15:49:59 +00:00
Ben Church	2d6f5ee88a	Dagster PoC for Metadata service (#23989) ... * Add airbyte-ci folders * Add poetry * Add first dagster job * Get sensors properly registering * Trigger job on new files * Add etag cursor * Wire up resources and ops * Parse destinations dataframe * Add multiple dataframes * Compute markdown * Write html and markdown * Move columns to variable * move to a folder centered file structure * Move to sensor factory * Add resource def to sensor * Use appropriate credentials * Use GCSFileManager * use catalog_directory_manager * Generalize the gcp catalog resources * Move bucket to env var * Clean up and add comments * Update readme * Move dependencies into orchestrator * Add gcs section to readme * Clean up debug * Add merge catalog tests * Run code formatter * Apply flake8 fixes * Remove suffix * Move tests up one level * Folder rename * Update readme and rename env * Add jinja templates * Rename connectors_ci to connectors for lib	2023-03-20 12:28:10 -07:00