Hypersolid

For a global trading firm, I designed and built the foundation of a data synchronization platform between their recruitment system and their marketing automation platform. The goal: one talent engagement pipeline, where marketing and recruitment share the same data instead of working in silos. The platform runs serverless on AWS, with Lambda functions and infrastructure as code in AWS CDK.

The assignment asked me to hit the ground running. I joined an existing codebase and AWS environment on a short timeline, and in about four weeks I delivered the foundation of the platform.

The sync had two hard parts. The first was keeping both sides consistent without endless update loops. I designed an incremental sync, with its state in DynamoDB and hash-based change detection, so only real changes are synchronized, and changes made by the sync itself are recognized and skipped.

The second was the business and synchronization rules, which were complex and not yet clear when I started. To get them clear fast, I worked them out with the business stakeholders as visual workflow diagrams. Those diagrams became the specification, and a code generator I built turns them into the structure of the code, so the implementation follows the flow that was agreed.

External APIs time out and rate-limit, so failure handling was part of the design. Requests are retried with exponential backoff on timeouts, 429 and 5xx responses. Other client errors fail immediately, and a client-side throttle keeps the sync within the marketing platform's rate limit. Errors are categorized, so the logs show whether a failure is a validation problem, a third-party outage or a bug. Because the platform handles candidate data, personal data is redacted from the logs. Metrics go to CloudWatch, with alarms on sync failures, error rates and Lambda errors, throttles and duration, and a dashboard that shows the health of every sync. Delivery was phased, starting with basic tagging before the full field synchronization.

As a Senior Backend Engineer, my responsibilities included:

• Designing a serverless synchronization architecture on AWS, from data model to monitoring
• Designing hash-based change detection that prevents update loops between two systems
• Building integrations with third-party recruitment and marketing APIs, with retries, backoff and rate limiting
• Getting up to speed quickly in an existing codebase and AWS environment
• Clarifying unclear requirements with business stakeholders through visual workflows, and generating code from them
• Developing Infrastructure as Code with AWS CDK
• Setting up CloudWatch alarms, dashboards, structured logging and PII redaction
• Writing documentation for developers, DevOps and stakeholders