Spectral Core Blog

Migration to Fabric

Migration to Fabric

Introduction Microsoft introduced Microsoft Fabric, an all-in-one SaaS analytics solution for enterprises that offers a comprehensive suite of services, including Data Factory, Data Engineering, Data Warehousing, Data Science, Real-Time Intelligence, and Power BI. This article focuses on our migration accelerator tools, Omni Loader and SQL Tran. These tools allow Synapse

Emulating identity columns in Fabric

Emulating identity columns in Fabric

Identity columns are present in practically every SQL Server database. But if you are modernizing your on-premises OLAP system to Fabric Warehouse, identity columns are still not supported. SQL Tran fixes this issue by analyzing your code and writing the equivalent. Of course, it is not as efficient as in

SQL Tran launch!

SQL Tran launch!

After many years in development, SQL Tran, a complete game-changer for database migration, is here. This is an end-to-end solution for SQL translation, capable of easily handling even the most complex codebases. There is no comparable offering on the market today: * Complete translation, which includes parsing and full static analysis,

500x faster than ANTLR

500x faster than ANTLR

If you, by any chance, read my article from half a year ago where I have introduced our HummingBird parser, you saw that it's incredibly fast, roughly 250x-300x faster than ANTLR. I did some relaxing and self-therapeutic coding this week, optimizing for .NET9. Not bad! We are now

10 million records per second, on-premises to the cloud

10 million records per second, on-premises to the cloud

By now, it is well known that Spectral Core database migration tooling is the most performant on the market, by some margin. Benchmarks would be good to back up such bold claims, but benchmarks are tricky and take time so let's use the next best thing - some

De-branding!

So... After a lot of preparation and with great expectations, Spectral Core rebranded to EQUEL last year. Fantastic name and fantastic domain we have used on and off for some 10 years now. What could go wrong? It turns out - several things. The most important realization is that I

Introducing Hummingbird SQL parser

Introducing Hummingbird SQL parser

SQL is notoriously difficult to parse. There is a standard, ANSI SQL - you shell out over $2000 to get your hands on the specs but get little value for all that money. Each database engine rolls out own extensions to the standard, and no database implements the whole standard

Spectral Core becomes Equel!

Spectral Core becomes Equel!

We are growing! Our press release follows: Spectral Core Unveils the Rebranding and Introduces Ultra-performing Products for Data Engineers Pula, Croatia – Spectral Core, a leading, fully independent provider of unique database tooling, is thrilled to announce a significant milestone with a comprehensive rebranding effort and a new product launch. Equel,

Online Database Migration (first look)

Online Database Migration (first look)

Our customers convert massive databases with Full Convert. We are rarely talking about less than a million records, and some customers are converting dozens of billions of records in a single run. People with small tables and databases can't afford to pay us $700-$3000 for a powerful

SQLite vs MySQL

SQLite vs MySQL

Context and useful information in regarding to some of the world's most popular databases. I hope this article will help you decide between SQLite and MySQL for your project.

Autocompleting SQL

Autocompleting SQL

I expect getting the SQL autocompletion behaving decently, in all conditions, is going to take a lot of time, and a lot of trial and error.

Mark and sweep

Efficiently collecting data changes for the very low-bandwidth distributed replication scenarios.

The process of writing... code.

The process of writing... code.

Writing a complex thing these days, so I thought of the process. Painting came to mind, which triggered memories of and old book, and here we are.

Hashing large tables

Hashing large tables

Diving into the hashing stage of our hash-based database comparison and replication.

Huge databases, tiny bandwidth

Huge databases, tiny bandwidth

Copying huge databases when bandwidth is small requires creative solutions.

You ain't gonna need it

You ain't gonna need it

YAGNI (you ain't gonna need it) principle is one of my favorites.

Pricing is tied to perception of you

Pricing is tied to perception of you

The way you price your offering is closely related to how people perceive your company and the offering. Impacting that perception allows you to change the pricing as well.

Diversification protects you

Diversification protects you

An unexpected event pulls you down. Was can be done to prepare for this?

Pitfall of over-engineering

Pitfall of over-engineering

My first successful product was built under huge time constraints (I had just a few hours a day, mostly late night) while I was under extreme pressures. So I built a minimal things that solves the problem. Turns out I forgot about that recipe once I had a lot of time to "build things right".

All database managers suck

All database managers suck

With databases powering practically every business in developed countries, one would expect tooling to be state of the art. There are a ton of tools, but state of the art?

Complexities of parsing SQL

Complexities of parsing SQL

What is so special about SQL? Why is it harder to parse and analyze compared to the other mainstream languages?

How does one parse source code?

How does one parse source code?

Source code is just a text. Transforming that text into a structured format needed for software to understand what is written is called parsing.

CDC, hacking for performance

CDC, hacking for performance

For those ready to ignore all the rules and go very low-level, the ultimate performance is the ultimate reward.

White-hat change data capture

White-hat change data capture

Certain database engines provide sanctioned ways to notify of changes to the data. That's better than polling or manual trigger creation.

Instrumenting production databases using triggers

Instrumenting production databases using triggers

In this short series of database replication techniques, we already covered polling for changes. If we are fine with attaching triggers on our tables, we can get much better response time.