BREAGEN is a comprehensive data management platform designed to streamline the process of managing data sources within an organization. By centralizing metadata and providing a unified platform for discovery, governance, and lineage, BREAGEN simplifies the challenges associated with understanding, accessing, and utilizing data assets.
Here's a brief overview of how BREAGEN can help manage data sources:
Discovery and Inventory: BREAGEN creates a comprehensive inventory of all data sources within the organization, making it easy to locate and understand their purpose, ownership, and usage.
Governance and Compliance: By enforcing data quality standards and ensuring adherence to compliance regulations, BREAGEN helps maintain data integrity and protect sensitive information.
Lineage and Impact Analysis: BREAGEN tracks the flow of data from source to consumption, enabling organizations to understand the potential impact of changes to data sources and identify dependencies.
Collaboration and Teamwork: BREAGEN fosters collaboration among data teams by providing a shared platform for communication and knowledge sharing.
BREAGEN supports a wide range of data sources, including:
Databases: MySQL, PostgreSQL, Oracle, SQL Server, etc.
Data Warehouses: Redshift, Snowflake, Teradata, etc.
Data Lakes: S3, ADLS, GCS, etc.
Streaming Platforms: Kafka, Kinesis, Pulsar, etc.
ETL/ELT Tools: Informatica, Talend, Airflow, etc.
Custom Sources: APIs, files, and other data formats.
By leveraging BREAGEN's capabilities, organizations can establish a robust data management framework, improve data quality, and drive data-driven decision-making.
Create a Dataset GET v1/data/managment?action=ingestProposal
Read Terms from a Dataset GET /v1/data/managment?action=getTerms&entityType=dataset&entityUrn=urn:li:dataset:(your_dataset_urn)
curl request to read the glossary terms for a dataset using the same API endpoint. Typically, this would involve a GET request to fetch the glossary terms associated with a particular dataset.
Create Domain GET /v1/data/managment?action=ingestProposal
curl request to read the glossary terms for a dataset using the same API endpoint. Typically, this would involve a GET request to fetch the glossary terms associated with a particular dataset.
curl --location --request POST 'https://api.braegen.ai/v1/data/managment?action=ingestProposal' \--header 'X-RestLi-Protocol-Version: 2.0.0' \--header 'Content-Type: application/json' \--data '{"proposal": { "entityType": "dataset", "entityUrn": "urn:li:dataset:(your_dataset_urn)", # Replace with your dataset URN "aspects": [ { "com.linkedin.metadata.domains.DomainAssociation": { "domainUrn": "urn:li:domain:(your_domain_urn)", # Replace with your domain URN "domainProperties": { "name": "BREAGEN Chat Domain", "description": "Domain for managing datasets within the BREAGEN Chat environment.", "created": { "time": 1694044800000, # Example timestamp for when the domain is assigned (Unix time in ms) "actor": "urn:li:corpuser:datafabadmin" # Replace with the user who is associating the domain } } } } ]}}'
curl --location --request POST 'https://api.braegen.ai/v1/data/managment?action=ingestProposal' \--header 'X-RestLi-Protocol-Version: 2.0.0' \--header 'Content-Type: application/json' \--data '{"proposal": { "entityType": "dataset", "entityUrn": "urn:li:dataset:(your_source_dataset_urn)", # Replace with the source dataset URN "aspects": [ { "com.linkedin.metadata.lineage.UpstreamLineage": { "upstreams": [ { "dataset": "urn:li:dataset:(your_target_dataset_urn)", # Replace with the target dataset URN "type": "TRANSFORMED", # Type of lineage (e.g., TRANSFORMED, COPIED, etc.) "created": { "time": 1694044800000, # Example timestamp (Unix time in ms) "actor": "urn:li:corpuser:datafabadmin" # Replace with the user performing the lineage creation }, "fieldPaths": [ "column_name_1", # Replace with the specific column(s) involved in the lineage "column_name_2" # Add more columns as needed ] } ] } } ]}}'
curl --location --request POST 'https://api.braegen.ai/v1/data/managment?action=ingestProposal' \--header 'X-RestLi-Protocol-Version: 2.0.0' \--header 'Content-Type: application/json' \--data '{"proposal": { "entityType": "dataset", "entityUrn": "urn:li:dataset:(your_dataset_urn)", # Replace with your dataset URN "aspects": [ { "com.linkedin.dataset.schema.EditableSchemaMetadata": { "editableSchemaFieldInfo": [ { "fieldPath": "column_name", # Replace with the specific column name "description": "This is the description of the column in the dataset." # Add your column description here } ] } } ]}}'
curl --location --request POST 'https://api.braegen.ai/v1/data/catalog?action=search' \--header 'X-RestLi-Protocol-Version: 2.0.0' \--header 'Content-Type: application/json' \--data '{ "input": "your_search_query", # Replace with your search query "entity": "dataset", # You can change this to "dataset", "table", "column", etc. "start": 0, "count": 10 # Number of results to return}'
