Set up automatic discovery
As part of the onboarding process, Port provides you with a set of self-service actions to create new services
, workloads
, environments
, users
, and teams
.
These actions involve manually selecting the components related to the new entity.
These are routine tasks that most organizations perform on a regular basis, which is why the manual process of using these actions is not efficient or scalable.
This guide will walk you through automating the process of creating and updating these entities in a way that suits your organization's standards.
Not sure what these entities mean? See their definitions here.
Choose discovery typeโ
Automatic discovery can be configured using one of two approaches:
- Define one of your external tools as a "source of truth" for the resources you want to ingest.
- Use metadata from your external tools (e.g. labels) to identify and update an entity in Port.
Both options require modifying the mapping configuration of the relevant integration.
How to modify a mapping configuration (click to expand)
- Go to the data sources page of your portal.
- Under "Exporters", find the relevant integration and click on it.
- A window will open, containing the mapping configuration. Use the editor in the bottom-left corner to update the configuration.
- Click on the "Save & Resync" button to save the changes and resync the integration.
Option 1: Define a source of truthโ
This approach is useful when you want to create entities of a specific type (e.g. services, environments, teams, users) based on resources from a specific external tool.
Full exampleโ
One example that works for many organizations is to define a Git repository as a source of truth for services
, and automatically create a new service
in Port for each repository in your Git provider.
To achieve this, we need to update the mapping configuration of the Git integration to include an entry for the service
blueprint.
Here is an example using the GitHub
integration:
- kind: repository
selector:
query: 'true'
teams: true
port:
entity:
mappings:
identifier: .full_name
title: .name
blueprint: '"githubRepository"'
properties:
readme: file://README.md
url: .html_url
defaultBranch: .default_branch
relations:
githubTeams: '[.teams[].id | tostring]'
- kind: repository
selector:
query: 'true'
port:
entity:
mappings:
identifier: .full_name
title: .full_name
blueprint: '"service"'
relations:
repository: .full_name
The first kind
block is the default mapping when installing the GitHub integration. The meaning of this block is:
For every repository in the GitHub organization, create a new githubRepository
entity in Port with the specified properties.
The second kind
block is the one we need to add. The meaning of this block is:
For every repository in the GitHub organization, create a new service
entity in Port, and relate it to the relevant githubRepository
entity.
With this approach, services
will always have a related repository upon creation, and can later be enriched with additional data from other assets in your catalog.
Additional examples by integrationโ
Just like the example above, the blocks in the examples below can be added to the mapping configuration of the relevant integration to automatically create entities in your catalog.
- Services
- Environments
- Workloads
- Users
- Teams
Common examples for resources that can be used as a source of truth for services
:
GitHub repository (click to expand)
- kind: repository
selector:
query: 'true'
port:
entity:
mappings:
identifier: .full_name
title: .full_name
blueprint: '"service"'
relations:
repository: .full_name
GitLab project (click to expand)
- kind: project
selector:
query: 'true'
port:
entity:
mappings:
identifier: .path_with_namespace | gsub(" "; "")
title: .name
blueprint: '"service"'
relations:
git_lab_repositry: .path_with_namespace | gsub(" "; "")
Bitbucket repository (click to expand)
- kind: repository
selector:
query: 'true'
port:
entity:
mappings:
identifier: .name
title: .name
blueprint: '"service"'
relations:
bitbucketRepository: .name
Azure DevOps repository (click to expand)
- kind: repository
selector:
query: 'true'
port:
entity:
mappings:
identifier: .project.name + "/" + .name | gsub(" "; "")
title: .name
blueprint: '"service"'
relations:
azureDevopsRepository: .project.name + "/" + .name | gsub(" "; "")
SonarQube project (click to expand)
- kind: projects_ga
selector:
query: 'true'
apiFilters:
qualifier:
- TRK
metrics:
- code_smells
- coverage
- bugs
- vulnerabilities
- duplicated_files
- security_hotspots
- new_violations
- new_coverage
- new_duplicated_lines_density
port:
entity:
mappings:
identifier: .key
title: .name
blueprint: '"service"'
relations:
sonar_project: .key
Snyk target (click to expand)
- kind: target
selector:
query: 'true'
port:
entity:
mappings:
identifier: .id
title: .attributes.display_name
blueprint: '"service"'
relations:
snyk_target: .id
PagerDuty service (click to expand)
- kind: services
selector:
query: 'true'
port:
entity:
mappings:
identifier: .id
title: .name
blueprint: '"service"'
relations:
pager_duty_service: .id
Common examples for resources that can be used as a source of truth for environments
:
Kubernetes cluster (click to expand)
- kind: v1/namespaces
selector:
query: .metadata.name | contains("kube-system")
port:
entity:
mappings:
- identifier: env.CLUSTER_NAME
title: env.CLUSTER_NAME
blueprint: '"environment"'
relations:
k8s_cluster: env.CLUSTER_NAME
ArgoCD cluster (click to expand)
- kind: cluster
selector:
query: 'true'
port:
entity:
mappings:
identifier: .name
title: .name
blueprint: '"environment"'
relations:
argo_cluster: .name
Common examples for resources that can be used as a source of truth for workloads
:
Datadog service (click to expand)
- kind: service
selector:
query: 'true'
port:
entity:
mappings:
identifier: .attributes.schema."dd-service"
title: .attributes.schema."dd-service"
blueprint: '"workload"'
relations:
datadog_service: .attributes.schema."dd-service"
Sentry project (click to expand)
- kind: project-tag
selector:
query: 'true'
tag: environment
port:
entity:
mappings:
identifier: .slug + "-" + .__tags.name
title: .name + "-" + .__tags.name
blueprint: '"workload"'
relations:
sentry_project: .slug + "-" + .__tags.name
Dynatrace entity (click to expand)
- kind: entity
selector:
query: 'true'
entityFields: firstSeenTms,lastSeenTms,tags
entityTypes:
- APPLICATION
- SERVICE
port:
entity:
mappings:
identifier: .entityId
title: .displayName
blueprint: '"workload"'
relations:
dynatrace_entity: .entityId
Kubernetes workload (click to expand)
- kind: apps/v1/deployments
selector:
query: .metadata.namespace | startswith("kube") | not
port:
entity:
mappings:
- identifier: >-
.metadata.name + "-Deployment-" + .metadata.namespace + "-" + env.CLUSTER_NAME
title: .metadata.name
blueprint: '"workload"'
relations:
k8s_workload: .metadata.name + "-Deployment-" + .metadata.namespace + "-" + env.CLUSTER_NAME
ArgoCD application (click to expand)
- kind: application
selector:
query: 'true'
port:
entity:
mappings:
identifier: .metadata.uid
title: .metadata.name
blueprint: '"workload"'
relations:
argo_application: .metadata.uid
Common examples for resources that can be used as a source of truth for users
:
GitHub user (click to expand)
- kind: user
selector:
query: 'true'
port:
entity:
mappings:
identifier: .login
title: .login
blueprint: '"_user"'
relations:
git_hub_user: .login
GitLab user (click to expand)
- kind: user
selector:
query: 'true'
port:
entity:
mappings:
identifier: .username
title: .username
blueprint: '"_user"'
Common examples for resources that can be used as a source of truth for teams
:
GitHub team (click to expand)
- kind: team
selector:
query: 'true'
port:
entity:
mappings:
identifier: .id | tostring
title: .name
blueprint: '"_team"'
relations:
git_hub_team: .id | tostring
GitLab team (click to expand)
- kind: group-with-members
selector:
query: 'true'
includeBotMembers: 'true'
port:
entity:
mappings:
identifier: .full_path
title: .name
blueprint: '"_team"'
relations:
gitlab_group: .full_path
Azure DevOps team (click to expand)
- kind: team
selector:
query: 'true'
port:
entity:
mappings:
identifier: .id
title: .name
blueprint: '"_team"'
relations:
azure_devops_team: .id
Option 2: Use predefined metadataโ
Another way to automatically update your catalog is to use data from your external tools to identify a specific entity in Port.
This approach is useful when you want to update entities of a specific type (e.g. services, environments, teams, users) based on a tag, label, or other piece of metadata in a specific external tool.
Note that this approach requires you to have a way to identify the Port entity that you want to update.
This is usually achieved by adding a tag or label to the resource in the external tool, with an indicative key (for example, prefixed with port-
).
Full exampleโ
After installing the ArgoCD
integration and ingesting our ArgoCD applications, we may want to automatically connect them to their corresponding service
entities.
To achieve this, we need to update the mapping configuration of the ArgoCD integration to include an entry for the service
blueprint.
This example assumes that each ArgoCD application has a label named portService
with the value being the identifier of the relevant service
entity in Port.
- kind: application
selector:
query: 'true'
port:
entity:
mappings:
identifier: .metadata.labels.portService
blueprint: '"service"'
relations:
argocdApplication: .metadata.uid
The meaning of this configuration is:
For every application ingested from ArgoCD, update the service
entity with the identifier matching the portService
label, relating it to this argocdApplication
entity.
Map by propertyโ
If the label's value is not an identifier, but some other property of the entity, you can use a query rule to find the relevant entity and update it.
For example, say each ArgoCD application has a label named portService
with the value being the title
of the service
entity in Port. We can use the following mapping configuration:
- kind: application
selector:
query: 'true'
port:
entity:
mappings:
identifier:
combinator: '"and"'
rules:
- operator: '"="'
property: '"$title"'
value: .metadata.labels.portService
blueprint: '"service"'
relations:
argocdApplication: .metadata.uid
The meaning of this configuration is:
For every application ingested from ArgoCD, update the service
entity with the title matching the portService
label, relating it to this argocdApplication
entity.