Skip to main content

Data Mart ETL - 2026-10-08

Data Mart Team
Visma Net ERP Data Mart

In this release:

Action recommended: where each incremental fetch starts​

Getting started used to recommend starting each incremental fetch from the most recent updatedInDataMartDateTime you received. That can miss changes. A run pages through the results by id, so a record that changes on a page you have already read is not returned again in that run, and a later page can carry a higher timestamp. A next run that starts from that timestamp skips the change for good. Any run that takes longer than a minute is exposed, including every initial import.

Update your integration as follows:

  1. Before the first request of each run, note the current time in UTC: this is the run's start time.
  2. Pass updatedInDataMart set to one minute before the start time of your previous run, with the same value on every page.
  3. Save the new start time only after every page is processed.

If your integration started from the most recent timestamp received, your copy may already miss some changes. To be sure it is complete, run a full reimport after you switch. See Incremental fetching for the full procedure.

Deleted records in incremental fetches​

When a record is deleted in Visma.net ERP, Data Mart marks it with deletedDatabaseRecord: true. Until this release, the deleted record kept the updatedInDataMartDateTime of its last update. An incremental fetch therefore never returned the deletion, and the record stayed in your copy until a full reimport.

Now the deleted record gets a new updatedInDataMartDateTime when Data Mart processes the deletion. To remove deleted records from your copy:

  1. Pass includeDeleted: true in your incremental fetch. It defaults to false, and without it deleted records are not returned.
  2. For each returned record with deletedDatabaseRecord: true, delete your copy, matched on the record's natural key.

Processing the same deletion again does not change the timestamp, so the deletion does not come back by itself in later fetches. A fetch that overlaps the previous one, as recommended above, can return it again; deleting a record you already removed has no effect. This applies to every query that has an includeDeleted argument.

Example: released supplier invoices​

When a prebooked supplier invoice is released, ERP deletes its lines and creates them again with new line numbers. A supplierDocumentLines fetch with includeDeleted: true now returns the old lines with deletedDatabaseRecord: true and the new lines as new records. Before this release the old lines stayed in your copy next to the new ones, so the line amounts were counted twice.

The deletion can arrive several minutes before the new lines. A fetch that runs in between sees the invoice without lines; the next fetch returns them. ERP can also write a line back under the same line number. That line is returned as deleted first and as a current record in a later fetch, so apply the records in the order you receive them.

GeneralLedgerBalances​

The 2026-09-28 release notes stated that a removed balance receives a new updatedInDataMartDateTime. That was true for the one-time cleanup in that release, but not for balances removed after it. From this release it is true for every removed balance. The forward queries now also return the group of a removed balance again in an incremental fetch, with the corrected timeline, as described in those notes.

Records deleted before this release​

Records deleted before this release still carry their old updatedInDataMartDateTime. We update the timestamp of these records once, on 2026-10-15. This date replaces the earlier announcement of "not before 2026-10-19". The update changes updatedInDataMartDateTime only, and only on records that are already deleted.

Your first incremental fetch with includeDeleted: true after the update returns these records once, so you can remove them from your copy without a full reimport.

  • Volume: that fetch can return many more records than usual. Retrieve all pages.
  • Deletions you already applied: some records are returned although you already removed them, for example after a full reimport. Deleting them again has no effect.
  • GeneralLedgerBalances is not part of this update. Every balance marked as deleted before this release was marked by the one-time cleanup announced in the 2026-09-28 release notes, which already gave it a new updatedInDataMartDateTime. An incremental fetch with includeDeleted: true has therefore already returned it, and the forward queries do not return any group again because of this update.
  • Without includeDeleted: true: your results do not change.

Timestamps set when records are written​

updatedInDataMartDateTime used to be set when Data Mart prepared a batch of records, before writing it. For large batches the write finished seconds or minutes later, so a record could appear with a timestamp that was already minutes old. A fetch that ran before the write finished did not return the record, and a later fetch starting after that timestamp skipped it for good.

Now each group of up to 1,000 records gets its timestamp right before it is written. Data Mart writes several batches at the same time, so a record can still be committed shortly after a record with a later timestamp. The one-minute overlap in the recommended action covers this. Match records on their natural key, so a record you receive twice replaces your copy.