* verify that webBackendRecreateSource has a workspace and source that match
* verify that webBackendRecreateDestination has a workspace and destination that match
* verify that workspaces match when creating connections and refactor boundary
* fmt
Introduce a better port abstraction whose primary purpose is to confirm that ports are released when the Kube Pod Process is closed.
This prevents issues like #4660
I'm also opening more ports so we can run at least 10 syncs in parallel.
* Do not create config directory in fs persistence construction
* Run kube acceptance test only for testing purpose
* Wait for config volume to be ready
* Move config volume wait for fs persistence construction
* Restore ci workflow
* Prune imports
Closes#4660 .
On further investigation, it turns out we were not releasing the source ports. This is because of how the Process abstraction works - waitFor calls close under the hood. We were only calling waitFor if the process was still alive. This is determined by the exitValue which comes from the Kubernetes pod's termination status. However, these ports are a local resource and no close calls means they were left dangling, leading to the behaviour we see today.
Explicitly call close after retrieving the exit value of the Kubernetes pod. This better follows traditional assumptions around Processes - if the process returns some exit value, it means all resources associated with that process have been cleaned up.
Also,
- add in a bunch of debug logging for the future.
- have better names for Kubernetes workers to make operations easier.
* Mixpanel initiation
* copied schemas and specs file from singer connector
* authentication and a few streams
* Added Funnels + FunnelsList
* Added example of funnel response
* added incremental Funnels stream with tests
* added Annotations, CohortMembers, Engage, Cohorts, Funnels
* added Revenue
* fixed formatting
* fixed variable names
* fixed cohort_members and updated export streams
* moved start_date and date checks into SourceMixpanel class
* added error handling
* added unit test, update docs and ci creds
* fix url base for export stream
* added full and incremental read for export stream
* updated acceptance tests, added limit correction based on number of streams, export cursor is stored in datatime string
* Funnel stream - added complex state which contains state for each funnel
* added attribution windows support and project timezone config
* fixed formatting
* added default timezone
* added dynamic schema generation for Engage and Export streams
* fixed formatting
* fixed ability to pass start_date in datetime format as well
* fixed ability to pass start_date in datetime format as well
* added additional_properties field for dynamic schemas. updates regex for start_date matching to support old config file
* fixed formatting
* export stream - convert all values to default type - string
* added schema ref
* added new properties for funnel stream
* fixed formatting in funnel schema
* added build related files
* update changelog
* fixed and added comments, renamed rate_limit variable
* fixed formatting
* changed normalization for reserved mixpanel attributes like $browser
* alphabetise spec fields
* added description about API limit handling
* updated comment
