Skip to content

Quick Start Guide

This guide will help you get the Iceberg Data Engineering Platform up and running in minutes.

Prerequisites

Before you begin, ensure you have the following installed:

  • Docker (version 20.10 or later)
  • Docker Compose (version 2.0 or later)
  • Git (for cloning the repository)

Installation Steps

1. Clone the Repository

git clone <repository-url>
cd iceberg_data_engineering

2. Start the Platform

Start all services with Docker Compose:

docker-compose up -d

This command will: - Download and build all required Docker images - Start all services in the correct order - Set up the data storage buckets - Initialize the databases

3. Verify Installation

Check that all services are running:

docker-compose ps

You should see all services with "Up" status.

4. Access the Services

Once all services are running, you can access:

Service URL Default Credentials
Documentation http://localhost:8000 -
Trino http://localhost:8080 trino://trino@trino:8080
Dagster http://localhost:3030 -
Apache Superset http://localhost:8088 admin/admin
MinIO Console http://localhost:9001 minioadmin/minioadmin
Hue http://localhost:8888 -
Apache Ranger http://localhost:6080 admin/admin
Spark Master http://localhost:8081 -

Next Steps

  1. Explore the Architecture: Read the Architecture Overview to understand the system design
  2. Run Your First Pipeline: Follow the Data Domains guides to understand the data ingestion process
  3. Monitor Operations: Use Dagster Orchestration to monitor and manage your data pipelines

Troubleshooting

Common Issues

Services not starting: Check Docker logs for specific errors: 

docker-compose logs <service-name>

Port conflicts: Ensure the required ports (8000, 8080, 3030, 8088, 9001, 8888, 6080, 8081) are available on your system.

Memory issues: Ensure Docker has at least 4GB of memory allocated.

Getting Help

  • Check the Development Guide for detailed setup instructions
  • Review the Deployment Guide for production deployment
  • Open an issue on GitHub for bugs or feature requests
Last update: October 3, 2025
Created: October 3, 2025