How Data Engineering Improves Business Intelligence and Reporting Systems

We embed ourselves in customer data, build working solutions, and iterate in production. That’s not a manifesto. It’s just how we work.

At Creative CFO, we’ve built our BI practice on a principle that most consulting firms talk about but very few actually follow through on. We don’t advise on data problems from a distance. We get into the data, build the solutions ourselves, and keep iterating until they actually work in production.

We do this in the world of financial operations, inventory management, and management reporting for mid-market South African firms. It’s not glamorous. But it’s where the real problems live.

We Don’t Interview Users. We Work Alongside Them.

Discovery matters. Understanding the problem before you start solving it is fundamental. We do discovery differently. We discover by doing the work. By getting into the data, sitting alongside the people who use it, and understanding the first principles behind every task and process through direct experience.

When a popular South African retail chain came to us with Cin7/Shopify inventory sync chaos, the discovery wasn’t a sprint that produced a slide deck. It was months spent in their SKU data. Building barcode cleanup scripts, fixing revenue account configurations, working through the same product master nightmares their ops team dealt with daily. We created Google Apps Scripts to assign barcodes at scale because that’s what the business actually needed. Not what they said they needed in the kickoff call.

When a leading pet brand had a warehouse theft incident, we didn’t facilitate a workshop about inventory controls. We rebuilt their stock status report with alt code enrichment and fixed the stock status logic ourselves. We were living in their data and could see exactly where the blind spots were.

When a well-known financial advisory business had AUM reconciliation breaking between two platforms, we didn’t write a requirements document. We took over the pipeline, got our hands dirty in timezone-misaligned refresh schedules and manual reconciliations back to data received from external platform providers. We fixed it ourselves.

THE PATTERN:
Every engagement follows the same rhythm. Embed, execute, iterate. Work with the customer’s data long enough that real progress gets made. Not progress on paper. Real progress.

Engineers Do the Selling

We don’t have a sales team. It started out of necessity, but increasingly it’s by design. In complex, mission-critical engagements, customers don’t need relationship brokers. They need someone who understands the work.

Our BI team lead handles customer calls about Cin7 Core drop-ship functionality. He preps technical deep-dives for manufacturing customers on demand forecasting. He manages difficult conversations while simultaneously debugging ELT pipelines. There’s no handoff between “sales” and “delivery” because the person doing the delivery is the sales process.

Navigating a challenging working relationship when you’re deeply committed to seeing a project through to its milestones is not easy. Bureaucracy and structural complexity can make even the most well-intentioned engagements hard to move forward. But it also means customers get something rare: authentic technical credibility without the performance theatre of traditional consulting.

When we came across how Palantir describes their Forward Deployed Software Engineer methodology, it was one of those moments where someone else had put language to something we’d been doing for years. The parallels were hard to ignore. But we didn’t copy their playbook. We’d already been running ours.

Instead of polished pitch decks, our conversations sound more like this:

“How fast can we get your Cin7 data clean enough to trust?”
“What’s blocking you from actually using this Power BI report?”
“Can you give us direct database access so we stop wasting time on exports?”

It’s really hard to get that kind of motivation from people who come from sales backgrounds rather than the work itself.

We Iterate with Working Code, Not Slide Decks

This is the part that makes every formally-trained product manager uncomfortable. We often experiment with live products instead of designs, mockups, or prototypes.

When a customer discovers a new need, we don’t spend weeks in discovery sprints. We build it. Share it. Subtract, add, rebuild, share again. Full-scale product iteration with real working code, often in daily cycles.

For one retail customer, we built an on-demand data pipeline using n8n webhooks, Python notebooks, and Excel Online. We didn’t prototype it. We shipped it, watched how they used it, and refined it based on real usage patterns.

For multiple customers, we’ve spun up dbt models, Python scripts, and n8n automation workflows faster than most consultancies can schedule a follow-up meeting.

The downside is obvious. You can waste engineering time on code no one uses. But the trick is this only works if your engineers can make serious, meaningful improvements in under 24 hours. You need the skill and experience to make writing code cheaper than most teams can run traditional experiments.

Building Compounding Value

The real unlock for any practice like ours is when the work you do for one customer makes the next engagement faster, sharper, and more valuable. When every project adds to a shared foundation rather than starting from scratch.

We’ve built automated meeting transcript processing that extracts insights across hundreds of customer conversations. That’s a growing knowledge base that should compound across engagements. We’ve got a standardised three-layer dbt architecture and a set of property models that form what we call the ontology layer, making each new customer implementation faster than the last. We are busy building an AI Agent Knowledge Base that captures deep domain expertise from years of partner adviser work, turning static documentation into something that can actually be queried and reused.

There’s a concept in platform thinking where every user’s activity makes the product better for the next user. Palantir’s tagging system is probably the best-known example of this in data infrastructure. We’re not there yet. Our patterns compound, but they compound through institutional knowledge and deliberate effort rather than automated deployment. That’s the gap we’re working to close.

The Strategic Shift

This is what our “Data Strategy and Advisory” service needs to become. Not more hours-based consulting, but the productised layer that makes each customer engagement improve the offering for the next one.

Building a Platform by Composing Solutions

We’re building a platform by composing best-of-breed tools into a cohesive architecture that gets stronger with every customer engagement.

We compose solutions from best-of-breed tools.

There are real trade-offs to this approach. More integration overhead, more surface area for failure. But we’re not locked into one vendor’s paradigm, we can pick the best tool for each customer context, and we can optimise costs aggressively. Our Airbyte migration away from Fivetran paid for itself immediately.

The key is that each new engagement strengthens the platform. The standardised dbt architecture, the ontology layer, the automation templates. Every customer-specific solution adds to a shared foundation that makes the next one faster to deploy and more valuable out of the box.

Where We Are. Where We’re Going.

We’re a small BI practice in Cape Town that’s built something genuinely different for the South African mid-market. Not by following a framework or buying into a methodology. By doing the work, over and over, until the approach took shape on its own.

Here’s what we’ve got:

Technical credibility. We’re not slideware merchants. We ship working code.

Customer intimacy. We’re not running two-week discovery sprints. We’re embedded for months.

Rapid iteration. We can build, deploy, and refine solutions faster than most firms can schedule meetings.

Pattern extraction discipline. We’re consciously building reusable IP from customer work.

Here’s what we’re building toward:

• The ontological layer taking shape across customers.
• Standardised property models and dbt architecture that work across retail, manufacturing, and financial services.
• An AI-powered knowledge base.
• Power BI optimisation playbooks & Custom React Front Ends.
• n8n automation templates.
• A maturity assessment framework that diagnoses data capability gaps.

The question isn’t whether we’re doing consulting or building product. We’re doing both, simultaneously, because that’s how you build products that actually solve the problems that matter.

The gap between where we are and where we need to be isn’t philosophical. It’s execution. And we’re working on it every day.