jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2026-01-07 09:05:45 -05:00
Code Issues Packages Projects Releases Wiki Activity

Publish stream status messages in CDK (#24994)

Browse Source 
* Publish stream status messages in CDK

* Automated Commit - Formatting Changes

* Convert to StreamDescriptor

* Automated Commit - Formatting Changes

* Bump to latest protocol model

* Automated Commit - Formatting Changes

* Bump protocol version

* Add tests for stream status message creation

* Formatting

* Formatting

* Fix failing test

* Actually emit state message

* Automated Commit - Formatting Changes

* Bump airbyte-protocol

* PR feedback

* Fix parameter input

* Correctly yield status message

* PR feedback

* Formatting

* Fix failing tests

* Automated Commit - Formatting Changes

* Revert accidental change

* Automated Change

* Replace STOPPED with COMPLETE/INCOMPLETE

* Update source-facebook-marketing changelog

* Revert "Update source-facebook-marketing changelog"

This reverts commit 709edb800c.

---------

Co-authored-by: jdpgrailsdev <jdpgrailsdev@users.noreply.github.com>
This commit is contained in:
Jonathan Pearlin
2023-04-26 10:30:36 -04:00
committed by GitHub
parent 3e6c8a37b0
commit 2ebfa459cf
9 changed files with 242 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
airbyte-cdk/python/airbyte_cdk/utils/stream_status_utils.py Normal file
View File

@@ -0,0 +1,36 @@
#
# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
#
from datetime import datetime
from airbyte_cdk.models import (
AirbyteMessage,
AirbyteStreamStatus,
AirbyteStreamStatusTraceMessage,
AirbyteTraceMessage,
ConfiguredAirbyteStream,
StreamDescriptor,
TraceType,
)
from airbyte_cdk.models import Type as MessageType
def as_airbyte_message(stream: ConfiguredAirbyteStream, current_status: AirbyteStreamStatus) -> AirbyteMessage:
"""
Builds an AirbyteStreamStatusTraceMessage for the provided stream
"""
now_millis = datetime.now().timestamp() * 1000.0
trace_message = AirbyteTraceMessage(
type=TraceType.STREAM_STATUS,
emitted_at=now_millis,
stream_status=AirbyteStreamStatusTraceMessage(
stream_descriptor=StreamDescriptor(name=stream.stream.name, namespace=stream.stream.namespace),
status=current_status,
),
)
return AirbyteMessage(type=MessageType.TRACE, trace=trace_message)