🗺️

Shield Data Map

Agent-based personal data discovery that automatically maps PII across your databases and filesystems — classifying it against DPDPA categories, assigning risk tiers, and linking it to the processing purposes in your consent notices. The output feeds directly into your RoPA and gives breach-response teams an accurate scope of what was potentially exposed.

Supported sources

Postgres, MySQL, Mongo, MSSQL, filesystem

~20 MB

Idle RAM

Runs within a 256 MB container limit

100%

Data locality

Raw values never leave your infrastructure

What problem it solves: DPDPA Section 8(3) requires you to erase data once its purpose is fulfilled — but you cannot erase what you have not mapped. Data Map answers the first question: what personal data do you actually hold, and where?

How it works

You deploy an agent on your infrastructure

A lightweight Docker container runs on a host that has network access to your databases. The agent never opens inbound ports — it makes outbound HTTPS calls to the platform only.

The agent scans sources locally

For each source, the agent connects with read-only credentials, discovers schemas/tables/columns, and samples up to 500 rows per column. It runs pattern recognition (regex + heuristics) against column names and sampled values to classify PII.

Only findings metadata is sent to the platform

The raw sample values are discarded immediately after classification. What reaches the platform is metadata only: schema name, table name, column name, detected PII type (e.g. "email", "aadhaar"), confidence score, and row count estimate. No cell content ever leaves your servers.

The platform builds your data map

Findings appear as Data Map Assets — each tagged with a DPDPA PII tier (Special Category / Sensitive / Personal) and linkable to processing purposes, consent notices, and RoPA entries. Breach-response teams can query "what personal data was in this source?" directly from the platform.

Supported sources

Source type	Config key	What the agent scans
PostgreSQL	postgresql	All schemas/tables/columns; read-only session enforced at the driver level.
MySQL / MariaDB	mysql	All databases/tables/columns; read-only confirmed via SHOW GRANTS before scanning.
MongoDB	mongodb	All collections; documents sampled (default 200 per collection).
SQL Server (MSSQL)	mssql	All schemas/tables/columns. Requires the :full Docker image (adds ODBC Driver 18).
Filesystem	filesystem	CSV, JSON, Excel, TSV, log, SQL dump files in a read-only mounted directory. Max 50 MB per file.

Coming soon (not yet available)

Oracle Database, SAP HANA, IBM DB2, Snowflake, BigQuery, Redshift, Amazon DynamoDB, and other cloud-native stores are on the roadmap but not supported today. See Coming soon for status.

Prerequisites

✓Docker: The agent is distributed as a Docker image (tar). Docker Desktop or Docker Engine is required on the host that runs the agent.
✓Outbound HTTPS: The host must be able to reach https://api.dpdpashield.in (port 443). No inbound ports need to be opened.
✓Network access to your databases: The agent connects to your DB hosts using the same credentials and network path as any application. VPN/peering/security-group rules must permit the connection.
✓Read-only DB credentials: Never use admin or write-capable credentials. For PostgreSQL, the agent enforces set_session(readonly=true) at the driver level. For others, a db_datareader-equivalent role is sufficient.
✓A named Docker volume (or host mount): The agent stores its registration state (agent.json) in /data. A named volume (dpdpa-agent-state) is required for restarts not to generate a new identity. See the Security model for why.

Resource footprint

~20 MB RAM idle. Peaks during a scan (column samples in memory) but stays well within a 256 MB container limit for typical sources.

Two image variants

:latest (246 MB) — Postgres, MySQL, Mongo, filesystem.
:full (261 MB) — adds MSSQL/ODBC Driver 18.

Deploy the agent

Path A — Docker Compose (recommended)

Download docker-compose.yml from the agent package, set your token, and run two commands.

docker-compose.yml

# docker-compose.yml (from the agent package)
services:
  register:                      # one-shot — exits after writing state
    image: dpdpa-shield-agent:latest
    command: ["agent", "register", "--token", "${DPDPA_TOKEN}"]
    environment:
      BASE_URL: "${DPDPA_BASE_URL:-https://dpdpashield.in}"
    volumes:
      - agent-state:/data

  agent:                         # long-lived daemon
    image: dpdpa-shield-agent:latest
    command: ["agent", "run"]
    restart: unless-stopped
    environment:
      BASE_URL: "${DPDPA_BASE_URL:-https://dpdpashield.in}"
      AGENT_INTERVAL: "60"
      AGENT_CONFIG: /data/scan.yaml
    volumes:
      - agent-state:/data         # REQUIRED — state volume
      # - /host/data:/scan/data:ro  # filesystem sources go here (:ro enforced)
    deploy:
      resources:
        limits:
          memory: 256m
          cpus: "0.5"

volumes:
  agent-state:

Then:

DPDPA_TOKEN=<your_token> docker compose run --rm register

After admin approval in the dashboard:

docker compose up -d agent

Path B — Plain Docker (step by step)

1Generate a registration token

In the dashboard: Data Map → Agents → Deploy New Agent. The token is one-time and expires in 24 hours. Copy it before leaving the page.

2Download the agent image

Download via the dashboard button, or with curl (replace the bearer token):

# Step 1: get a short-lived presigned download URL (authenticated)
RESP=$(curl -s -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  "https://api.dpdpashield.in/api/v1/data-map/agent-image/download")

# Step 2: extract the URL and download directly from S3 (no auth needed for this step)
DOWNLOAD_URL=$(echo "$RESP" | python3 -c "import json,sys; print(json.load(sys.stdin)['data']['downloadUrl'])")
curl -L "$DOWNLOAD_URL" -o dpdpa-shield-agent-1.0.0.tar

3Verify integrity

sha256sum dpdpa-shield-agent-1.0.0.tar
# Compare the output to the SHA-256 shown in the dashboard Deploy Agent flow

The expected SHA-256 is shown in the dashboard Deploy Agent flow next to the download button.

4Load the image into Docker

docker load -i dpdpa-shield-agent-1.0.0.tar

5Register the agent (one-time)

Replace <YOUR_REGISTRATION_TOKEN> with the token from step 1:

docker run --rm \
  -v dpdpa-agent-state:/data \
  -e BASE_URL=https://dpdpashield.in \
  dpdpa-shield-agent:1.0.0 \
  agent register --token <YOUR_REGISTRATION_TOKEN>

The container exits after writing credentials to the dpdpa-agent-state volume. The agent appears in the dashboard with status PENDING.

6Approve the agent

In the dashboard: Data Map → Agents → find the PENDING agent → Approve. A DPO or Super Admin must approve before the agent can claim scan jobs. This gate exists so a stray or misplaced agent cannot start scanning without explicit authorisation.

7Start the daemon

docker run -d --restart unless-stopped \
  -v dpdpa-agent-state:/data \
  -e BASE_URL=https://dpdpashield.in \
  -e AGENT_CONFIG=/data/scan.yaml \
  dpdpa-shield-agent:1.0.0 \
  agent run

The daemon heartbeats every 60 seconds, picks up scan jobs queued from the dashboard, and posts findings back to the platform. The agent status changes to ONLINE.

+Adding filesystem sources (optional)

Mount host directories into the container as read-only volumes (:ro). The agent can read the files; kernel-level enforcement prevents writes.

docker run -d --restart unless-stopped \
  -v dpdpa-agent-state:/data \
  -v /your/data/directory:/scan/data:ro \
  -e BASE_URL=https://dpdpashield.in \
  -e AGENT_CONFIG=/data/scan.yaml \
  dpdpa-shield-agent:1.0.0 \
  agent run

Configure sources

Sources are defined in scan.yaml and stored on the state volume (/data/scan.yaml inside the container). The platform_source_id field binds each local source entry to the corresponding DataMapSource record in the platform — get the ID from the platform dashboard.

scan.yaml

# scan.yaml — stored at /data/scan.yaml inside the container
# Credentials stay on YOUR infrastructure. Only findings metadata reaches the platform.

sources:
  databases:
    - label: "Production PostgreSQL"
      type: postgresql           # connector key: "postgresql"
      platform_source_id: <id from dashboard>
      host: your-db-host
      port: 5432
      database: your_db
      username: readonly_user
      password: "${PG_PASSWORD}"   # use env vars — never hardcode passwords
      schemas: ["public"]

    - label: "Analytics MySQL"
      type: mysql                # connector key: "mysql"
      platform_source_id: <id from dashboard>
      host: 10.0.1.20
      port: 3306
      database: analytics_db
      username: scanner_ro
      password: "${MYSQL_PASSWORD}"

    - label: "User MongoDB"
      type: mongodb              # connector key: "mongodb"
      platform_source_id: <id from dashboard>
      uri: "mongodb://scanner:${MONGO_PASSWORD}@localhost:27017"
      database: users_db

    - label: "CRM SQL Server"
      type: mssql                # connector key: "mssql" — use the :full image
      platform_source_id: <id from dashboard>
      host: 192.168.1.50
      port: 1433
      database: crm_prod
      username: sa_readonly
      password: "${MSSQL_PASSWORD}"

Use read-only credentials. The agent performs SELECT queries only. For PostgreSQL it sets SET SESSION CHARACTERISTICS AS TRANSACTION READ ONLYand verifies SHOW transaction_read_only = on before scanning. For MySQL it checks GRANTS and aborts if INSERT/UPDATE/DELETE permissions are present.

Security model

For security reviewers and enterprise procurement teams. Each claim below maps to a specific implementation in the agent or platform codebase.

🔒 Data locality — raw values never leave your infrastructure

The scanner samples up to 500 row values per column for pattern matching. These values are local variables that go out of scope immediately after classification runs. What is transmitted to the platform is metadata only: schema/table/column name, detected PII type, confidence score (0–1), and row count estimate. No cell content, no sample values, no hashed user identifiers.

🛡️ Least privilege — read-only everywhere

• The container runs as a non-root user (uid=999, agent:agent).
• Filesystem mounts use the :ro flag — write attempts return EROFS: Read-only file system at the kernel level.
• Database connections enforce read-only mode at the driver level (PostgreSQL) or verify GRANTS before scanning (MySQL, MSSQL).
• The agent makes outbound HTTPS connections only. No inbound ports are opened.

🔑 Agent authorisation — multi-step, revocable

• Registration token: One-time, expires in 24 hours. Cannot be reused after the first register call (consumed atomically).
• PENDING → approval gate: Every new agent starts in PENDING status. A DPO or Super Admin must approve it before it can claim scan jobs. An agent that registers without approval cannot scan.
• Tenant-scoped: Each agent is bound to exactly one tenant. Agents cannot access data across tenant boundaries — all scan jobs and findings are filtered by tenantId.
• Revocable: An admin can revoke an agent from the dashboard at any time. Revoked agents receive a 401 on the next heartbeat and stop operating immediately.
• HMAC request signing: Every post-registration API call is signed with HMAC-SHA256(agentSecret, timestamp.rawBody). Requests with stale timestamps (±5 minutes) are rejected.

🧬 Credential-copy detection — environment fingerprinting

At registration the agent computes an environment fingerprint:SHA-256(instanceId + ":" + platformId)[:32], where platformId is the host's machine-id (Linux), IOPlatformUUID (macOS), or MachineGuid registry key (Windows). The fingerprint is stored on the platform and echoed on every heartbeat.

• Legitimate restart (same volume): instanceId is preserved → same fingerprint → no alert.
• Copied to another host: different platformId → different fingerprint → SERIOUS alert raised in the platform and flagged on the agent card. The agent continues operating (not auto-revoked by default) but the DPO is notified immediately.
• Auto-revoke opt-in: Setting AGENT_FINGERPRINT_AUTO_REVOKE=true on the platform will auto-revoke on mismatch. Default is off — a false positive on legitimate infra changes must not silently kill scanning.

⏱️ Phone-home expiry — forgiving, reversible self-deactivation

The agent tracks its last successful platform contact in agent.json. If the agent cannot reach the platform for an extended period, it pauses scanning rather than operating disconnected indefinitely.

• After 14 days offline: the platform raises an INFO alert — “agent has not checked in for 14 days; will pause after 21 days.”
• After 19 days offline: the platform escalates to SERIOUS — “agent will pause soon.”
• After 21 days offline: the agent daemon self-deactivates (stops claiming scan jobs) and logs clearly: “Agent paused: no platform contact for 21 days. Will resume automatically when connectivity is restored.”
• Fully reversible: on next successful contact the agent reactivates automatically. No re-registration, no manual intervention.
• Forgiving: brief outages (network blips, maintenance windows) below the 21-day threshold do not affect operation.

Troubleshooting

Agent stuck in PENDING

Likely cause: Registration succeeded but no DPO/Super Admin has approved it yet.

Fix: Go to Data Map → Agents in the dashboard and click Approve.

Agent shows OFFLINE or stops heartbeating

Likely cause: The host cannot reach https://api.dpdpashield.in (port 443 outbound blocked, or network partition).

Fix: Check outbound firewall rules from the agent host. The agent will auto-recover to ONLINE on the next successful heartbeat — no restart needed.

Scan returns 0 tables / 0 findings

Likely cause: Wrong database key in scan.yaml (dbname vs database), wrong schema name, or the scan.yaml is not mounted at /data/scan.yaml.

Fix: Verify the AGENT_CONFIG env var points to /data/scan.yaml. Check that the database: key matches the real database name. Check that schemas: lists the correct schema.

Permission check failed — connection is NOT read-only

Likely cause: The DB user has write permissions (INSERT/UPDATE/DELETE).

Fix: Create a dedicated read-only user. For PostgreSQL: GRANT SELECT ON ALL TABLES IN SCHEMA public TO scanner_ro. For MySQL: GRANT SELECT ON mydb.* TO scanner_ro.

Agent fingerprint mismatch flag on the dashboard

Likely cause: The agent.json state volume was used on a different host, or the agent was copied without its volume.

Fix: If the agent legitimately moved hosts (e.g. infra migration), revoke the flagged agent and re-register it from the new host. The flag is informational — the agent keeps working unless AGENT_FINGERPRINT_AUTO_REVOKE is set.

Authentication failed (401) — agent exits

Likely cause: The agent has been revoked from the dashboard, or the agentSecret was rotated.

Fix: A 401 means the agent credential is invalid. Delete the state volume and re-register with a new token.

docker load: manifest unknown

Likely cause: The downloaded tar is corrupted or incomplete.

Fix: Verify the SHA-256 of the downloaded file against the value shown in the dashboard. Re-download if they differ.

Coming soon

The following are on the roadmap but not yet available. They are listed here honestly so you can plan — not as present capabilities.

○Oracle Database connector

○SAP HANA connector

○IBM DB2 connector

○Snowflake, BigQuery, Amazon Redshift (cloud-native warehouses)

○Amazon DynamoDB

○S3 / object-store file scanning

○Automatic platform_source_id binding (agent self-registers its sources on first scan)

○Scheduled recurring scans with drift alerts (schema changes, new PII columns)

Contact hello@dpdpashield.in if a specific connector is blocking your deployment.