CDI Data Engineer

Day rate of up to £400
12 month contract with BRITISH PETROLEUM

About Our Client

Our client operates within the Oil and Gas sector. British Petroleum.

Job Description

Role 1: CDI Data Engineer

Implement and maintain SCI base, fact and curated pipelines.
Deliver Elavon/PAR/WRE/payment and loyalty matching.
Support ICE-to-EV data engineering and dashboard datasets.
Implement DO-site and market-specific logic.
Investigate volume drops, match-rate changes and data completeness defects.
Execute controlled reloads, production validation, DQ/freshness controls and BAU support.
Prepare PRs, tests, technical designs, CAB/release evidence and stakeholder

Role 2: BNA Production-Readiness, MLOps & Integration Engineer:

Automate end-to-end BNA orchestration
Reliable retry/recovery and idempotent processing
Production monitoring and alerting automated activation and acknowledgement handling
Reducing manual intervention Stable QA → production deployment process
Runbooks and operational handover
Reducing runtime and production failures

Role 3: CIS/SCI Integration & Post-Migration Stabilisation Engineer:

Repointing remaining CIS/SCI dependencies
Stabilising migrated pipelines
Reusable error-handling patterns
Reducing legacy technical debt
Removing dual-platform monitoring
Resolving SFMC/Comarch integration issues
Reducing production support burden
Clear handover into BAU

The Successful Applicant

Role 1: CDI Data Engineer

Must Have worked with:

Strong SQL, Python/Spark and Databricks pipeline engineering background.
Large-scale transaction, payment, loyalty, EPOS/WRE and identity-matching work history
Historical/full/incremental loads, high-water marks and reconciliation.
Azure DevOps PRs, unit testing, CI/CD and production release support.

Nice to have: Reporting/Power BI dataset exposure desirable. SFMC/CIS integration and multi-market SCI knowledge desirable.Role 2: BNA Production-Readiness, MLOps & Integration Engineer

Must Have:

Strong hands-on background in Databricks, PySpark, Python and SQL Production pipeline engineering and troubleshooting
Workflow orchestration and dependency management Idempotency, retry/restart and failure-recovery patterns CI/CD, version control and automated deployment
Worked across monitoring, logging and operational alerting
Completed data-quality and reconciliation controls API/file/SFTP integration patterns

Nice to have: Delta Lake / Delta MERGE Databricks Asset Bundles Azure DevOps MLOps/model deployment work history SFMC or Comarch integration Customer decisioning / personalisation platformsRole 3: CIS/SCI Integration & Post-Migration Stabilisation Engineer

Must Have worked with:

Strong hands-on background with Databricks, SQL, Python/PySpark ETL/ELT Data-pipeline migration
Legacy-to-cloud modernisation
Production troubleshooting
Dependency analysis
Pipeline repointing
Error handling and recovery
Data validation/reconciliation
Integration testing
Production support / L2-L3 troubleshooting

Nice to have: AWS-to-Databricks migration Informatica / SSIS / legacy ETL SFMC Comarch SFTP/file interfaces Azure/AWS CI/CD