GitLab
GitLab
GitLab ETL connector for data replication
Snapshot
| Features | Details |
| Release Status | In-dev |
| Source API Version | V4 |
| Table Selection | Yes |
| Column Selection | Yes |
| Edit Integration | Yes |
| Replication Type Selection | No |
| Authentication Parameters | GitLab Company Name GitLab API Key |
| Replication Type | Full Replication Key Based Incremental |
| Replication Key | Date |
| Suggested Replication Frequency | 1 hour |
Tables/APIs supported
| Projects | Groups |
| Epics | Project Milestones |
| Group Milestones | Releases |
| Tags | Todos |
| Issues | Comments |
| Commits | Merge Requests |
| Branches | Users |
Integrate GitLab with Daton
- Sign in to Daton
- Select GitLab from the list of Integrations
- Provide Integration Name, Replication Frequency, and History. Integration name would be used in creating tables for the integration and cannot be changed later
- Provide your GitLab company name and API Key
- Post successful authentication, you will be prompted to choose from the list of available tables
- Then select all required fields for each table
- Submit the integration
Workflow
- Integrations would be in Pending state initially and will be moved to Active state as soon as the first job loads data successfully on to the configured warehouse
- Users would be able to edit/pause/re-activate/delete integration anytime
- Users can view job status and process logs from the integration details page by clicking on the integration name from the active list
GitLab Data
Projects
Purpose: This API is used to fetch all the projects in your GitLab Account
Source API Documentation: https://docs.gitlab.com/ee/api/projects.html
Replication: Key Based Incremental
Replication Key: last_activity_after, last_activity_before
Fields
| Name | Data Type |
| id | NUMERIC |
| description | STRING |
| name | STRING |
| name_with_namespace | STRING |
| path | STRING |
| path_with_namespace | STRING |
| created_at | TIMESTAMP |
| default_branch | STRING |
| tag_list | STRING |
| topics | STRING |
| ssh_url_to_repo | STRING |
| http_url_to_repo | STRING |
| web_url | STRING |
| readme_url | STRING |
| avatar_url | STRING |
| forks_count | NUMERIC |
| star_count | NUMERIC |
| last_activity_at | TIMESTAMP |
| namespace | RECORD |
Groups
Purpose: This API is used to fetch all the groups
Source API Documentation: https://docs.gitlab.com/ee/api/groups.html
Replication: Full Replication
Replication Key: NA
Fields
| Name | Data Type |
| id | NUMERIC |
| name | STRING |
| path | STRING |
| description | STRING |
| visibility | STRING |
| share_with_group_lock | BOOLEAN |
| require_two_factor_authentication | BOOLEAN |
| two_factor_grace_period | NUMERIC |
| project_creation_level | STRING |
| subgroup_creation_level | STRING |
| lfs_enabled | BOOLEAN |
| default_branch_protection | NUMERIC |
| avatar_url | STRING |
| web_url | STRING |
| request_access_enabled | BOOLEAN |
| full_name | STRING |
| full_path | STRING |
| file_template_project_id | NUMERIC |
| created_at | TIMESTAMP |
Epics (GitLab Premium Users Only)
Purpose: This API is used to fetch all the Epics under the groups. This is available only to GitLab Premium Self-managed, GitLab
Premium SaaS and higher tiers
Source API Documentation: https://docs.gitlab.com/ee/api/epics.html
Replication: Key Based Incremental
Replication Key: updated_after, updated_before
Fields
| Name | Data Type |
| id | NUMERIC |
| iid | NUMERIC |
| group_id | NUMERIC |
| parent_id | NUMERIC |
| parent_iid | NUMERIC |
| title | STRING |
| description | STRING |
| state | STRING |
| confidential | STRING |
| web_url | STRING |
| reference | STRING |
| references | RECORD |
| author | RECORD |
| start_date_is_fixed | BOOLEAN |
| end_date | STRING |
| due_date | STRING |
| due_date_is_fixed | BOOLEAN |
| due_date_from_milestones | STRING |
| due_date_from_inherited_source | STRING |
| created_at | TIMESTAMP |
| updated_at | TIMESTAMP |
| closed_at | TIMESTAMP |
| upvotes | NUMERIC |
| downvotes | NUMERIC |
| color | STRING |
| _links | RECORD |
Project Milestones
Purpose: This API is used to fetch the milestones of all the projects
Source API Documentation: https://docs.gitlab.com/ee/api/milestones.html
Replication: Key Based Incremental
Replication Key: last_activity_after, last_activity_before
Fields
| Name | Data Type |
| id | NUMERIC |
| iid | NUMERIC |
| project_id | NUMERIC |
| title | STRING |
| description | STRING |
| due_date | DATE |
| start_date | DATE |
| state | STRING |
| updated_at | TIMESTAMP |
| created_at | TIMESTAMP |
| expired | BOOLEAN |
Group Milestones
Purpose: This API is used to fetch milestones of all the groups
Source API Documentation: https://docs.gitlab.com/ee/api/group_milestones.html
Replication: Full Replication
Replication Key: NA
Fields
| Name | Data Type |
| id | NUMERIC |
| iid | NUMERIC |
| group_id | NUMERIC |
| title | STRING |
| description | STRING |
| due_date | DATE |
| start_date | DATE |
| state | STRING |
| updated_at | TIMESTAMP |
| created_at | TIMESTAMP |
| expired | BOOLEAN |
| web_url | STRING |
Releases
Purpose: This API is used to fetch the releases project wise
Source API Documentation: https://docs.gitlab.com/ee/api/releases/
Replication: Key Based Incremental
Replication Key: last_activity_after, last_activity_before
Fields
| Name | Data Type |
| tag_name | STRING |
| description | STRING |
| name | STRING |
| created_at | TIMESTAMP |
| released_at | TIMESTAMP |
| author | RECORD |
| commit | RECORD |
| milestones | RECORD |
| commit_path | STRING |
| tag_path | STRING |
| assets | RECORD |
| evidences | RECORD |
Tags
Purpose: This API is used to get a list of repository tags from a project
Source API Documentation: https://docs.gitlab.com/ee/api/tags.html
Replication: Key Based Incremental
Replication Key: last_activity_after, last_activity_before
Fields
| Name | Data Type |
| commit | RECORD |
| release | RECORD |
| name | STRING |
| target | STRING |
| protected | BOOLEAN |
| project_id | STRING |
Todos
Purpose: This API is used to fetch all to-dos that are done and pending
Source API Documentation: https://docs.gitlab.com/ee/api/todos.html
Replication: Full Replication
Replication Key: NA
Fields
| Name | Data Type |
| id | NUMERIC |
| project | RECORD |
| author | RECORD |
| action_name | STRING |
| target_type | STRING |
| target | RECORD |
| target_url | STRING |
| body | STRING |
| state | STRING |
| created_at | TIMESTAMP |
| updated_at | TIMESTAMP |
Issues
Purpose: This API is used to fetch all the issues in the account
Source API Documentation: https://docs.gitlab.com/ee/api/issues.html
Replication: Key Based Incremental
Replication Key: updated_after, updated_before
Fields
| Name | Data Type |
| state | STRING |
| description | STRING |
| author | RECORD |
| milestone | RECORD |
| project_id | NUMERIC |
| assignees | RECORD |
| assignee | RECORD |
| type | STRING |
| updated_at | TIMESTAMP |
| id | NUMERIC |
| title | STRING |
| created_at | TIMESTAMP |
| iid | NUMERIC |
| labels | STRING |
| upvotes | NUMERIC |
| downvotes | NUMERIC |
| merge_requests_count | NUMERIC |
| user_notes_count | NUMERIC |
| due_date | DATE |
| web_url | STRING |
| references | RECORD |
| time_stats | RECORD |
| has_tasks | BOOLEAN |
| task_status | STRING |
| confidential | BOOLEAN |
| discussion_locked | BOOLEAN |
| issue_type | STRING |
| severity | STRING |
| _links | RECORD |
| task_completion_status | RECORD |
Comments
Purpose: This API is used to fetch all comments in issues, epics, snippets and merge requests
Source API Documentation: https://docs.gitlab.com/ee/api/notes.html
Replication: Key Based Incremental
Replication Key: updated_after, updated_before
Fields
| Name | Data Type |
| id | NUMERIC |
| body | STRING |
| author | RECORD |
| created_at | TIMESTAMP |
| updated_at | TIMESTAMP |
| system | BOOLEAN |
| noteable_id | NUMERIC |
| noteable_type | STRING |
| noteable_iid | NUMERIC |
| resolvable | BOOLEAN |
| confidential | BOOLEAN |
| internal | BOOLEAN |
Commits
Purpose: This API is used to fetch the commits made in all the projects. In commit responses, created_at and commited_date are identical.
Source API Documentation: https://docs.gitlab.com/ee/api/commits.html
Replication: Key Based Incremental
Replication Key: since, until
Fields
| Name | Data Type |
| id | STRING |
| short_id | STRING |
| title | STRING |
| author_name | STRING |
| author_email | STRING |
| authored_date | TIMESTAMP |
| committer_name | STRING |
| committer_email | STRING |
| committed_date | TIMESTAMP |
| created_at | TIMESTAMP |
| message | STRING |
| parent_ids | STRING |
| web_url | STRING |
| project_id | STRING |
Merge Requests
Purpose: This API is used to fetch all the merge requests made by all users
Source API Documentation: https://docs.gitlab.com/ee/api/merge_requests.html
Replication: Key Based Incremental
Replication Key: updated_after, updated_before
Fields
| Name | Data Type |
| id | NUMERIC |
| iid | NUMERIC |
| project_id | NUMERIC |
| title | STRING |
| description | STRING |
| state | STRING |
| merged_by | RECORD |
| merge_user | RECORD |
| merged_at | TIMESTAMP |
| created_at | TIMESTAMP |
| updated_at | TIMESTAMP |
| target_branch | STRING |
| source_branch | STRING |
| upvotes | NUMERIC |
| downvotes | NUMERIC |
| author | RECORD |
| assignee | RECORD |
| assignees | RECORD |
| reviewers | RECORD |
| source_project_id | NUMERIC |
| target_project_id | NUMERIC |
| labels | STRING |
| draft | BOOLEAN |
| work_in_progress | BOOLEAN |
| milestone | RECORD |
| merge_when_pipeline_succeeds | BOOLEAN |
| merge_status | STRING |
| detailed_merge_status | STRING |
| sha | STRING |
| user_notes_count | NUMERIC |
| should_remove_source_branch | BOOLEAN |
| force_remove_source_branch | BOOLEAN |
| allow_collaboration | BOOLEAN |
| allow_maintainer_to_push | BOOLEAN |
| web_url | STRING |
| references | RECORD |
| time_stats | RECORD |
| squash | BOOLEAN |
| task_completion_status | RECORD |
Branches
Purpose: This API is used to fetch branches under all projects
Source API Documentation: https://docs.gitlab.com/ee/api/branches.html
Replication: Full Replication
Replication Key: NA
Fields
| Name | Data Type |
| name | STRING |
| merged | BOOLEAN |
| protected | BOOLEAN |
| default | BOOLEAN |
| developers_can_push | BOOLEAN |
| developers_can_merge | BOOLEAN |
| can_push | BOOLEAN |
| web_url | STRING |
| commit | RECORD |
| project_id | STRING |
Users
Purpose: This API is used to fetch all users
Source API Documentation: https://docs.gitlab.com/ee/api/users.html
Replication: Full Replication
Replication Key: NA
Fields
| Name | Data Type |
| id | NUMERIC |
| username | STRING |
| name | STRING |
| state | STRING |
| avatar_url | STRING |
| web_url | STRING |
