Skip to content

Commit dfc3d9c

Browse files
fivetran-avinashgithub-actions[bot]fivetran-savage
authored
[Feature] Add Linkedin share history (#35)
* Changelog * Changelog * bring in Linkedin Pages seed updates, version bump * documentation * Generate dbt docs via GitHub Actions * schema * remove unnecessary column types * PR review updates * Generate dbt docs via GitHub Actions * CHANGELOG + integration_tests/dbt_project.yml updates * CHANGELOG * Update models/intermediate/social_media_reporting__linkedin_posts_reporting.sql Co-authored-by: Savage Fivetran <sarah.savage@fivetran.com> * Update packages.yml * packages.yml * Generate dbt docs via GitHub Actions --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Savage Fivetran <sarah.savage@fivetran.com>
1 parent 7f3bd86 commit dfc3d9c

12 files changed

Lines changed: 96 additions & 10 deletions

‎CHANGELOG.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,25 @@
1+
# dbt_social_media_reporting v1.5.0
2+
[PR #35](https://github.com/fivetran/dbt_social_media_reporting/pull/35) includes the following updates:
3+
4+
## Schema/Data Change **(--full-refresh required after upgrading)**
5+
**21 total changes • 13 of which are breaking in the form of materialization updates**
6+
7+
| Data Model(s) | Change type | Old | New | Notes |
8+
| ---------- | ----------- | -------- | -------- | ----- |
9+
| `stg_linkedin_pages__organization`<br> `stg_linkedin_pages__organization_ugc_post`<br> `stg_linkedin_pages__post_content`<br> `stg_linkedin_pages__share_statistic`<br> `stg_linkedin_pages__ugc_post_history`<br> `stg_linkedin_pages__ugc_post_share_statistic`<br> (as well as the upstream tmp models for each model) | Materialization | Table | View | **Breaking:** These staging models are now views. Upon full refresh, the package will automatically drop the tables and recreate them as views. However, we recommend reviewing your schema and manually dropping any remaining staging models materialized as tables as they will no longer be updated in favor of the materialized views. |
10+
| `linkedin_pages__posts` | Renamed column | `ugc_post_id` | `post_id` | **Breaking:** Column renamed to reflect that the model now includes both UGC posts and shares. |
11+
| `social_media_reporting__rollup_report` | Data expansion | UGC posts only | UGC posts + shares | Linkedin Pages data will now includes both UGC posts and LinkedIn shares. |
12+
| `social_media_reporting__linkedin_posts_reporting` | Data expansion | UGC posts only | UGC posts + shares | Linkedin Pages data will now includes both UGC posts and LinkedIn shares. |
13+
| `linkedin_pages__posts` | Data expansion | UGC posts only | UGC posts + shares | Model now includes both UGC posts and LinkedIn shares. |
14+
| `linkedin_pages__posts` | Added column | | `content_type` | Indicates whether the post is from 'ugc' (UGC post) or 'share' (LinkedIn share). |
15+
| `stg_linkedin_pages__organization_share` | New model | | | Mapping table between organizations and shares. |
16+
| `stg_linkedin_pages__share_content` | New model | | | Content of shares (articles, images, polls, text). |
17+
| `stg_linkedin_pages__share_history` | New model | | | Version history of shares with metadata. |
18+
| `stg_linkedin_pages__share_share_statistic` | New model | | | Mapping between shares and share statistics. |
19+
20+
## Under the Hood
21+
- Added seed files for new LinkedIn data sources, as well as designed column types to ensure no data type confusion errors.
22+
123
# dbt_social_media_reporting v1.4.0
224

325
[PR #34](https://github.com/fivetran/dbt_social_media_reporting/pull/34) includes the following updates:

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ Each Quickstart transformation job run materializes the following model counts f
6767
| Social Media Reporting | 6 |
6868
| [Facebook Pages](https://github.com/fivetran/dbt_facebook_pages) | 11 |
6969
| [Instagram Business](https://github.com/fivetran/dbt_instagram_business) | 7 |
70-
| [LinkedIn Company Pages](https://github.com/fivetran/dbt_linkedin_pages) | 15 |
70+
| [LinkedIn Company Pages](https://github.com/fivetran/dbt_linkedin_pages) | 23 |
7171
| [Twitter Organic](https://github.com/fivetran/dbt_twitter_organic) | 11 |
7272
| [Youtube Analytics](https://github.com/fivetran/dbt_youtube_analytics) | 11 |
7373

@@ -103,7 +103,7 @@ Include the following github package version in your `packages.yml`
103103
```yaml
104104
packages:
105105
- package: fivetran/social_media_reporting
106-
version: [">=1.4.0", "<1.5.0"] # we recommend using ranges to capture non-breaking changes automatically
106+
version: [">=1.5.0", "<1.6.0"] # we recommend using ranges to capture non-breaking changes automatically
107107
```
108108
Do NOT include the upstream social media packages in this file. The transformation package itself has a dependency on it and will install the upstream packages as well.
109109
@@ -230,7 +230,7 @@ packages:
230230
version: [">=1.1.0", "<1.2.0"]
231231
232232
- package: fivetran/linkedin_pages
233-
version: [">=1.1.0", "<1.2.0"]
233+
version: [">=1.3.0", "<1.4.0"]
234234
235235
- package: fivetran/youtube_analytics
236236
version: [">=1.1.0", "<1.2.0"]

‎dbt_project.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
name: 'social_media_reporting'
2-
version: '1.4.0'
2+
version: '1.5.0'
33
config-version: 2
44
require-dbt-version: [">=1.3.0", "<3.0.0"]
55
vars:

‎docs/catalog.json‎

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

‎docs/manifest.json‎

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

‎integration_tests/dbt_project.yml‎

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
name: 'social_media_rollup_integration_tests'
2-
version: '1.4.0'
2+
version: '1.5.0'
33
profile: 'integration_tests'
44
config-version: 2
55

@@ -18,6 +18,10 @@ vars:
1818
linkedin_pages_post_content_identifier: "linkedin_pages_post_content_data"
1919
linkedin_pages_ugc_post_history_identifier: "linkedin_pages_ugc_post_history_data"
2020
linkedin_pages_ugc_post_share_statistic_identifier: "linkedin_pages_ugc_post_share_statistic_data"
21+
linkedin_pages_organization_share_identifier: "linkedin_pages_organization_share_data"
22+
linkedin_pages_share_content_identifier: "linkedin_pages_share_content_data"
23+
linkedin_pages_share_history_identifier: "linkedin_pages_share_history_data"
24+
linkedin_pages_share_share_statistic_identifier: "linkedin_pages_share_share_statistic_data"
2125
instagram_business:
2226
instagram_business_schema: social_media_rollup_integration_tests
2327
instagram_business_media_history_identifier: "instagram_business_media_history_data"
@@ -69,7 +73,23 @@ seeds:
6973
linkedin_pages_ugc_post_share_statistic_data:
7074
+column_types:
7175
ugc_post_id: "{%- if target.type == 'bigquery' -%} INT64 {%- else -%} bigint {%- endif -%}"
72-
76+
linkedin_pages_share_share_statistic_data:
77+
+column_types:
78+
share_id: "{%- if target.type == 'bigquery' -%} INT64 {%- else -%} bigint {%- endif -%}"
79+
linkedin_pages_organization_share_data:
80+
+column_types:
81+
share_id: "{%- if target.type == 'bigquery' -%} INT64 {%- else -%} bigint {%- endif -%}"
82+
linkedin_pages_share_content_data:
83+
+column_types:
84+
share_id: "{%- if target.type == 'bigquery' -%} INT64 {%- else -%} bigint {%- endif -%}"
85+
linkedin_pages_ugc_post_history_data:
86+
+column_types:
87+
created_actor: "{%- if target.type == 'bigquery' -%} STRING {%- elif target.type == 'databricks' -%} string {%- else -%} varchar {%- endif -%}"
88+
last_modified_actor: "{%- if target.type == 'bigquery' -%} STRING {%- elif target.type == 'databricks' -%} string {%- else -%} varchar {%- endif -%}"
89+
linkedin_pages_share_history_data:
90+
+column_types:
91+
id: "{%- if target.type == 'bigquery' -%} INT64 {%- else -%} bigint {%- endif -%}"
92+
7393
instagram_business_media_history_data:
7494
+column_types:
7595
id: "{%- if target.type == 'bigquery' -%} INT64 {%- else -%} bigint {%- endif -%}"

‎integration_tests/seeds/linkedin_pages_organization_share_data.csv‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
_fivetran_synced,organization_id,share_id
2+
2021-10-27T14:40:09.984,10429,5975571345575022592
3+
2021-10-27T14:40:09.984,10429,5994461776131948545
4+
2021-10-27T14:40:09.984,10429,5986812882107527168
5+
2021-10-27T14:40:10.560,10429,1035074525
6+
2021-10-27T14:40:10.560,10429,1034687017
7+
2021-10-27T14:40:09.333,10429,6213067404298190848
8+
2021-10-27T14:40:08.757,3954657,5881690557952073728
9+
2021-10-27T14:40:08.757,3954657,5875271880520269824
10+
2021-10-27T14:40:08.757,3954657,5879163220291264512
11+
2021-10-27T14:40:08.757,3954657,5882381493124222977

‎integration_tests/seeds/linkedin_pages_share_content_data.csv‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
_fivetran_synced,_fivetran_id,share_id,article_description,article_source,article_thumbnail,article_thumbnail_alt_text,article_title,carousel_id,media_alt_text,media_id,media_title,multi_image_alt_text,poll_question,poll_settings_duration,poll_settings_is_voter_visible_to_author,poll_settings_vote_selection_type,poll_unique_voters_count,type
2+
2023-10-16 17:40:48.712000,A1b2C3d4E5f6G7h8I9j0K1l2M3o=,5975571345575022592,Check out our latest product update,https://example.com/article1,https://example.com/thumb1.jpg,Product thumbnail,New Feature Release,,,,,,,,,,0,article
3+
2023-10-16 17:40:48.817000,B2c3D4e5F6g7H8i9J0k1L2m3N4p=,5994461776131948545,,,,,,,Images of team meeting,urn:li:digitalmediaAsset:C4E0BAQHEOJHNqMopbA,Team Photo,,,,,,0,multi_image
4+
2023-10-16 17:40:48.711000,C3d4E5f6G7h8I9j0K1l2M3n4O5q=,5986812882107527168,,,,,,,,,,,What's your favorite feature?,P14D,true,SINGLE_VOTE,125,poll
5+
2023-10-16 17:40:49.702000,D4e5F6g7H8i9J0k1L2m3N4o5P6r=,1035074525,,,,,,,,,,,,,,,0,text
6+
2023-10-16 17:40:49.701000,E5f6G7h8I9j0K1l2M3n4O5p6Q7s=,1034687017,Join our webinar series,https://example.com/webinar,https://example.com/webinar-thumb.jpg,Webinar banner,Upcoming Webinar Series,,,,,,,,,,0,article
7+
2023-10-16 17:40:49.712000,F6g7H8i9J0k1L2m3N4o5P6q7R8t=,6213067404298190848,,,,,,,Office photo,urn:li:digitalmediaAsset:C560BAQHLfJsj_Db26Q,New Office,,,,,,0,image
8+
2023-10-16 17:40:48.710000,G7h8I9j0K1l2M3n4O5p6Q7r8S9u=,5881690557952073728,,,,,,,,,,,,,,,0,text
9+
2023-10-16 17:40:51.730000,H8i9J0k1L2m3N4o5P6q7R8s9T0v=,5875271880520269824,Industry insights and trends,https://example.com/insights,https://example.com/insights-thumb.jpg,,Industry Report 2023,,,,,,,,,,0,article
10+
2023-10-16 17:40:51.731000,I9j0K1l2M3n4O5p6Q7r8S9t0U1w=,5879163220291264512,,,,,,,Product demo screenshot,urn:li:digitalmediaAsset:C561BAQEvDoEtu3nA4Q,Product Demo,Image showing new interface,,,,,0,image
11+
2023-10-16 17:40:51.731000,J0k1L2m3N4o5P6q7R8s9T0u1V2x=,5882381493124222977,,,,,,,,,,,How do you use our product?,P7D,true,MULTIPLE_VOTE,87,poll

‎integration_tests/seeds/linkedin_pages_share_history_data.csv‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
_fivetran_synced,author,commentary,container_entity,created_actor,created_time,distribution_external_distribution_channels,distribution_feed_distribution,first_published_at,id,last_modified_actor,last_modified_time,lifecycle_state,response_context_parent,response_context_root,visibility
2+
2023-10-16 17:40:48.712000,urn:li:organization:10429,uDfL4SB+lHgin1DHtu6udg==,,urn:li:person:12345,2023-10-13 17:32:47.678000,[],NONE,2023-10-13 17:32:47.678000,5975571345575022592,urn:li:person:12345,2023-10-13 17:32:47.732000,PUBLISHED,,,PUBLIC
3+
2023-10-16 17:40:48.817000,urn:li:organization:10429,KsVVYNtNduP4F2Bl/3I7+w==,,urn:li:person:12345,2023-10-05 22:00:04.871000,[],NONE,2023-10-05 22:00:04.871000,5994461776131948545,urn:li:person:12345,2023-10-05 22:00:04.919000,PUBLISHED,,,PUBLIC
4+
2023-10-16 17:40:48.711000,urn:li:organization:10429,VjYsdz1afLT4LZv27tpOxw==,,urn:li:person:12345,2023-10-13 17:34:23.935000,[],NONE,2023-10-13 17:34:23.935000,5986812882107527168,urn:li:person:12345,2023-10-13 17:34:23.983000,PUBLISHED,,,PUBLIC
5+
2023-10-16 17:40:49.702000,urn:li:organization:10429,8/uRr/D6X8E38FuzKyI5gQ==,,urn:li:person:12345,2023-09-14 02:45:58.968000,[],NONE,2023-09-14 02:45:58.968000,1035074525,urn:li:person:12345,2023-09-14 02:45:59.036000,PUBLISHED,,,PUBLIC
6+
2023-10-16 17:40:49.701000,urn:li:organization:10429,96qtGOk0mY4ddEOzGOy4Cw==,,urn:li:person:12345,2023-09-14 02:46:31.268000,[],NONE,2023-09-14 02:46:31.268000,1034687017,urn:li:person:12345,2023-09-14 02:46:31.319000,PUBLISHED,,,PUBLIC
7+
2023-10-16 17:40:49.712000,urn:li:organization:10429,nS+zf/EDU/nq9r9cyuQxkw==,,urn:li:person:12345,2023-09-12 23:02:00.434000,[],NONE,2023-09-12 23:02:00.434000,6213067404298190848,urn:li:person:12345,2023-09-12 23:02:46.136000,PUBLISHED,,,PUBLIC
8+
2023-10-16 17:40:48.710000,urn:li:organization:3954657,uDfL4SB+lHgin1DHtu6udg==,,urn:li:person:67890,2023-10-13 17:36:54.411000,[],NONE,2023-10-13 17:36:54.411000,5881690557952073728,urn:li:person:67890,2023-10-13 17:36:54.457000,PUBLISHED,,,PUBLIC
9+
2023-10-16 17:40:51.730000,urn:li:organization:3954657,KsVVYNtNduP4F2Bl/3I7+w==,,urn:li:person:67890,2023-07-31 15:57:07.634000,[],NONE,2023-07-31 15:57:07.634000,5875271880520269824,urn:li:person:67890,2023-07-31 15:57:07.692000,PUBLISHED,,,PUBLIC
10+
2023-10-16 17:40:51.731000,urn:li:organization:3954657,VjYsdz1afLT4LZv27tpOxw==,,urn:li:person:67890,2023-07-31 15:56:35.705000,[],NONE,2023-07-31 15:56:35.705000,5879163220291264512,urn:li:person:67890,2023-07-31 15:56:35.754000,PUBLISHED,,,PUBLIC
11+
2023-10-16 17:40:51.731000,urn:li:organization:3954657,8/uRr/D6X8E38FuzKyI5gQ==,,urn:li:person:67890,2023-07-30 14:22:18.123000,[],NONE,2023-07-30 14:22:18.123000,5882381493124222977,urn:li:person:67890,2023-07-30 14:22:18.456000,PUBLISHED,,,PUBLIC

‎integration_tests/seeds/linkedin_pages_share_share_statistic_data.csv‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
_fivetran_synced,share_id,share_statistic_id
2+
2021-10-27T14:40:09.984,5975571345575022592,K1l2M3n4O5p6Q7r8S9t0U1v2W3x=
3+
2021-10-27T14:40:09.984,5994461776131948545,L2m3N4o5P6q7R8s9T0u1V2w3X4y=
4+
2021-10-27T14:40:09.984,5986812882107527168,M3n4O5p6Q7r8S9t0U1v2W3x4Y5z=
5+
2021-10-27T14:40:10.560,1035074525,N4o5P6q7R8s9T0u1V2w3X4y5Z6a=
6+
2021-10-27T14:40:10.560,1034687017,O5p6Q7r8S9t0U1v2W3x4Y5z6A7b=
7+
2021-10-27T14:40:09.333,6213067404298190848,P6q7R8s9T0u1V2w3X4y5Z6a7B8c=
8+
2021-10-27T14:40:08.757,5881690557952073728,Q7r8S9t0U1v2W3x4Y5z6A7b8C9d=
9+
2021-10-27T14:40:08.757,5875271880520269824,R8s9T0u1V2w3X4y5Z6a7B8c9D0e=
10+
2021-10-27T14:40:08.757,5879163220291264512,S9t0U1v2W3x4Y5z6A7b8C9d0E1f=
11+
2021-10-27T14:40:08.757,5882381493124222977,T0u1V2w3X4y5Z6a7B8c9D0e1F2g=

0 commit comments

Comments
 (0)