keytouse/ubl_zatca repository overview

⁠ZATCA-compliant E-Invoicing — JSON to UBL XML

A tool for generating ZATCA-compliant UBL XML invoices from JSON, signing them, and submitting them to the Fatoora gateway. Full onboarding, signing, submission, and audit logging — all over HTTP.

• Hosted endpoint: https://ubl.keytouse.com⁠
• Source code: github.com/jaami/ubl_zatca⁠
• Self-hosted stack: this Docker image

⁠Two ways to use it — best together

If all you need is JSON → XML, the hosted endpoint alone is enough.

If you want the full pipeline — onboarding, signing, submission, and audit — deploy this image.

The full solution uses both. The local stack orchestrates everything; the hosted endpoint performs the stateless XML conversion.

⁠Verified against

⁠Architecture

ZATCA compliance has two separate concerns:

⁠Remote endpoint

1. Receives the invoice JSON
2. Deletes any prior invoice (single-invoice design, globally)
3. Generates the UBL XML
4. Returns it

The remote holds one invoice globally at any time. Each new upload replaces the previous one.

⁠Local stack

1. Receives a JSON invoice (file upload)
2. Validates it against the required schema
3. Stores it in a local MySQL database (permanent history)
4. Sends the same JSON to the remote for XML generation
5. Signs the returned XML with your CSID (XAdES-BES)
6. Submits the signed invoice to the Fatoora gateway
7. Records the response in SubmissionLog

Your invoice data persists locally. The remote sees only the in-flight payload.

⁠Quick Start (hosted)

Send your JSON invoice to the hosted endpoint and save the XML:

curl -X POST https://ubl.keytouse.com/zatca/xml \
  -H "Content-Type: application/json" \
  -d @your_invoice.json \
  -o invoice.xml

The request body must be a list containing one object with requesttype: "setdata" and a data field. See Sample JSON payload below.

⁠Quick Start (self-hosted)

⁠Prerequisites

• Docker or Podman
• ~2 GB free disk space
• Outbound HTTPS access to https://ubl.keytouse.com and gw-fatoora.zatca.gov.sa

⁠1. Create a working directory

mkdir zatca-local && cd zatca-local

⁠2. Extract the application files

docker run --rm --entrypoint tar \
  keytouse/ubl_zatca:ktuzatca-flask-app-beta-1.1 \
  -cf - -C /app \
  --exclude=jdk11 --exclude=ZatcaSDK --exclude=__pycache__ \
  . | tar -xf -

This copies app.py, zatca_client.py, InsertIntoDB.py, DeleteInvoice.py, xmlToSDK.py, required_invoice_elements.json, schema.sql, and supporting files into the current directory.

⁠3. Create docker-compose.yml

services:
  flask-app:
    image: keytouse/ubl_zatca:ktuzatca-flask-app-beta-1.1
    container_name: ktuzatca-flask-app
    ports:
      - "5000:5000"
    environment:
      - ZATCA_ENV=simulation
    volumes:
      - ./app.py:/app/app.py:ro
      - ./zatca_client.py:/app/zatca_client.py:ro
      - ./InsertIntoDB.py:/app/InsertIntoDB.py:ro
      - ./DeleteInvoice.py:/app/DeleteInvoice.py:ro
      - ./xmlToSDK.py:/app/xmlToSDK.py:ro
      - ./required_invoice_elements.json:/app/required_invoice_elements.json:ro
      - ./credentials:/app/credentials
      - ./credentials/cert.pem:/app/ZatcaSDK/Data/Certificates/cert.pem
      - ./credentials/private-key.pem:/app/ZatcaSDK/Data/Certificates/ec-secp256k1-priv-key.pem
    networks:
      - my_network
    depends_on:
      - mysql

  mysql:
    image: keytouse/ubl_zatca:mysql-8.0.39
    container_name: ktuzatca-mysql-1
    environment:
      MYSQL_ROOT_PASSWORD: TheRoot@Pass
      MYSQL_DATABASE: keytouse_zatca
    volumes:
      - mysql_data:/var/lib/mysql
      - ./schema.sql:/docker-entrypoint-initdb.d/01-schema.sql:ro
    networks:
      - my_network

networks:
  my_network:
    driver: bridge

volumes:
  mysql_data: {}

Before deploying, change MYSQL_ROOT_PASSWORD to your own value. Port 3306 is intentionally not exposed to the host — the database is reachable only from inside the Docker network.

⁠4. Start the stack

docker compose up -d

Wait ~15 seconds for MySQL to initialize and load the schema.

⁠5. Verify

curl http://localhost:5000/hi
# -> Hello there!

⁠The full pipeline (Simulation onboarding)

From nothing to a Production CSID, all over HTTP:

⁠Stage 1 — Generate a CSR and private key

curl -X POST http://localhost:5000/zatca/onboard/csr \
  -H "Content-Type: application/json" \
  -d '{"env": "simulation", "config": "/app/credentials/csr-config-simulation.properties"}'

Returns the CSR path and key path. The key is saved to the mounted credentials/private-key.pem so it survives container restarts.

Each environment uses a different certificate template inside the CSR:

⁠Stage 2 — Exchange CSR + OTP for a Compliance CSID

curl -X POST http://localhost:5000/zatca/onboard/compliance \
  -H "Content-Type: application/json" \
  -d '{"otp": "<YOUR_OTP>", "env": "simulation"}'

Requires an OTP from the Fatoora portal's Onboarding section. Returns a requestID, binarySecurityToken, and secret. The Compliance CSID is valid for 24 hours.

⁠Stage 3 — Extract the certificate

ZATCA returns the certificate double-encoded. Decode once and save to the mounted cert.pem:

python3 << 'EOF'
import json, base64
d = json.load(open('credentials/compliance_csid.json'))
inner = base64.b64decode(d['binarySecurityToken']).decode('utf-8')
with open('credentials/cert.pem', 'w') as f:
    f.write(inner)
print("cert.pem:", len(inner), "bytes")
EOF

⁠Stage 4 — Run the six compliance checks

ZATCA requires six test invoices before issuing a Production CSID:

For each: set the two fields in invoice.json, then:

curl -X POST -F "[email protected]" http://localhost:5000/upload
curl -X POST http://localhost:5000/zatca/submit \
  -H "Content-Type: application/json" \
  -d '{"invoiceid": "INV001"}'

Expected: status: "PASS" with clearance: "CLEARED" (Standard) or reporting: "REPORTED" (Simplified).

Credit and Debit notes require cac:BillingReference and cac:PaymentMeans/cbc:InstructionNote. The remote generator adds these automatically based on InvoiceTypeCode.

⁠Stage 5 — Request the Production CSID

curl -X POST http://localhost:5000/zatca/onboard/production \
  -H "Content-Type: application/json" \
  -d '{"env": "simulation"}'

Returns a new binarySecurityToken and secret. Saved to credentials/production_csid.json.

The route writes only the cert — the private key is not touched. ZATCA issues the Production CSID for the same key pair as the CCSID.

⁠Stage 6 — Submit with the Production CSID

/zatca/submit auto-detects the cert type: uses production_csid.json if present, otherwise compliance_csid.json. No code change needed.

curl -X POST -F "[email protected]" http://localhost:5000/upload
curl -X POST http://localhost:5000/zatca/submit \
  -H "Content-Type: application/json" \
  -d '{"invoiceid": "INV001"}'

Every submission is recorded in SubmissionLog:

podman exec -i ktuzatca-mysql-1 mysql -uroot -pTheRoot@Pass -t keytouse_zatca << 'EOF'
SELECT ID, InvoiceID, Environment, HTTPStatus, ZATCAStatus, ClearanceStatus, SubmittedAt
FROM SubmissionLog ORDER BY ID DESC LIMIT 5;
EOF

⁠Production onboarding

Everything above runs against Simulation without affecting real tax records. Production onboarding is separate:

1. Repeat Stage 1 with "env": "production"
2. Repeat Stage 2 with a Production OTP
3. The 6 compliance checks become real legal filings
4. Request the Production CSID — valid for real invoicing

Production onboarding affects the taxpayer's tax record. It requires explicit consent.

⁠API Reference (local stack)

⁠Invoice and submission

⁠Onboarding

All routes are HTTP — no terminal or SDK knowledge required.

⁠API Reference (hosted endpoint)

All requests are POST with a JSON body. The body is a list containing one request object.

⁠Request types

⁠Sample JSON payload

The data field must contain a Config array and all invoice sections.

[
  {
    "requesttype": "setdata",
    "data": {
      "Config": [
        {
          "InvoiceType": "StandardInvoice",
          "CreatedBy": "192.168.1.10",
          "Status": "NEW"
        }
      ],
      "Invoice": [
        {
          "ID": "INV001",
          "ProfileID": "reporting:1.0",
          "UUID": "8e6f8f2b-6e6a-4a4a-9c1c-1234567890ab",
          "IssueDate": "2026-09-16",
          "IssueTime": "14:30:00",
          "InvoiceTypeCode": "388",
          "InvoiceTypeName": "0100000",
          "Note": "Sample invoice",
          "DocumentCurrencyCode": "SAR",
          "TaxCurrencyCode": "SAR"
        }
      ],
      "AccountingParty": [ /* supplier + customer rows */ ],
      "AccountingPartyAddress": [ /* addresses */ ],
      "AccountingPartyTaxScheme": [ /* VAT numbers */ ],
      "AdditionalDocumentReference": [ /* ICV, PIH, QR */ ],
      "Signature": [ /* signature metadata */ ],
      "Delivery": [ /* ... */ ],
      "PaymentMeans": [ /* ... */ ],
      "AllowanceCharge": [ /* ... */ ],
      "AllowanceTaxCategory": [ /* ... */ ],
      "AllowanceTaxCategoryScheme": [ /* ... */ ],
      "TaxTotal": [ /* ... */ ],
      "TaxSubTotal": [ /* ... */ ],
      "TaxSubCategory": [ /* ... */ ],
      "TaxSubCategoryScheme": [ /* ... */ ],
      "LegalMonetaryTotal": [ /* ... */ ],
      "InvoiceLine": [ /* ... */ ],
      "InvoiceLineTaxTotal": [ /* ... */ ],
      "InvoiceLineItem": [ /* ... */ ],
      "InvoiceLineTaxCategory": [ /* ... */ ],
      "InvoiceLineTaxScheme": [ /* ... */ ],
      "InvoiceLinePrice": [ /* ... */ ],
      "InvoiceLineAllowanceCharge": [ /* ... */ ]
    }
  }
]

⁠Rules worth knowing

1. PaymentMeans.AccountingPartyID must reference the customer, not the supplier. Otherwise:

1EQ: Error: PaymentMeans should have at least one row.

2. When TaxCurrencyCode is present, the server adds a second bare TaxTotal automatically. You can send just one — the server duplicates it as a bare block. BR-KSA-EN16931-09 compliant without any client-side work.

3. Supplier VAT must be 15 digits, starting and ending with 3. Set AccountingPartyTaxScheme.CompanyID accordingly. Example: 311111111111113. Otherwise:

BR-KSA-40: seller VAT registration number must contain 15 digits.
The first and the last digits are "3".

4. Credit Note (381) and Debit Note (383) require a reason for issuance. The remote generator adds <cac:BillingReference> and <cac:PaymentMeans><cbc:InstructionNote> automatically from the invoice's Note field.

⁠Errors and status codes

Duplicate handling. ZATCA returns 208 (Clearance) or 409 (Reporting) if the same invoice hash is submitted within 24 hours. Both are success indicators — the invoice was accepted on the first submission. The reportingStatus: NOT_REPORTED field in a 409 body is a response artifact, not a failure. The pipeline records both as DUPLICATE_CLEARED / DUPLICATE_REPORTED, never as errors.

⁠Local ZATCA validation

After signing, the pipeline runs the XML through the official ZATCA SDK (238-R3.3.8) using the bundled fatoora CLI. It validates:

• XML schema (XSD)
• English content rules (EN)
• Saudi Arabia-specific rules (KSA)
• QR code
• Signature
• Previous Invoice Hash (PIH)

If validation fails, the SDK names the exact rule that was violated.

⁠Database schema

The MySQL image ships with a MariaDB database named keytouse_zatca. Tables map directly to the JSON sections, plus:

• SubmissionLog — every submission with full request and response
• IPLog — audit trail of every request by client IP
• InvoiceType — allowed invoice type codes (388, 389, 381, 383, ...)

⁠Concurrency model

Requests that touch the invoice tables are serialized with a MySQL named lock (zatca_invoice_lock). If two clients submit at the same moment, the second receives 429 Server busy immediately — no blocking, no waiting.

⁠Configuration

The /zatca/submit route auto-detects cert type: uses production_csid.json if present, otherwise compliance_csid.json.

⁠What this project is not

• Not a ZATCA SDK replacement. Signing, canonicalisation, and validation use the official SDK.
• Not multi-tenant. The remote is single-invoice.
• Not a billing system. No invoicing history beyond the local database.
• Not tied to Saudi Arabia only in design. The UBL layer is generic; ZATCA-specific rules are what this repo implements.

⁠Roadmap

• Auto-generate the second TaxTotal on the remote (removes client-side awareness)
• In-memory XML generation (removes DB from the hot path — no lock needed)
• Per-invoice storage on the remote (allows multiple concurrent invoices)
• Additional country adapters (Finland, Peppol, India GST)
• Server-side invoice signature

⁠Changelog

⁠2026-09-27

• Full Simulation onboarding verified: Compliance CSID obtained, all 6 compliance checks passed, Production CSID issued, production clearance endpoint returns CLEARED.
• Credit Note (381) and Debit Note (383) support with BillingReference and InstructionNote (BR-KSA-56 and BR-KSA-17).
• Auto-detect cert type in /zatca/submit.
• generate_csr() copies the private key to a stable mounted path and refuses to run if a CCSID already exists.
• load_csid() selects cert file by cert type (compliance vs production).
• README rewritten with the full pipeline walkthrough.

⁠2026-09-20

• New tag ktuzatca-flask-app-beta-1.1 — layers current source over beta-1.0.
• Added zatca_client.py, /zatca/submit, /zatca/onboard/* routes.
• Added SubmissionLog table.
• Fixed UTF-8 handling, remote error surfacing, retry/timeout.
• Verified end-to-end against Sandbox.

⁠2024-12-29

• Invoice generation and local validation via fatoora CLI verified.

⁠2024-12-24

• Local validation switched to the command-line fatoora tool.

⁠Feedback

[email protected]⁠

Thanks!

𝓎𝓪𝓼𝓲𝓻𝓲𝓶𝓻𝓪𝔶