Search This Blog

Sunday, November 3, 2019

Fixing date field formats to upgrade elastic to version 7

Here is what did, that eventually worked:


  1. The only way I found to EDIT the field format was by creating a new (empty) index, by specifying its mapping. The date fields I replaced (had my json in a text editor and did a "replace all" on each different format to a elastic-7-complying one. In my case, most of my fields were "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis", and I changed them to "8yyyy/MM/dd HH:mm:ss||8yyyy/MM/dd".
  2. Then I issued a _reindex, meaning I copied the contents of each index I had to fix to its corresponding new empty index I created, in the step above, to hold its data, now in a compatible format
  3. At the end of the reindex (do not worry about the "backend connection closed" message, the copy keeps running, check the "Reload Indices" in Index Management), I added an alias to the new index so that new documents being added to the original one are now inserted in the new one (using the _alias api)
  4. If you want the indexes to remain with their original names at the end, you can then copy once again the indexes to a new one, using the original name this time. Be aware that before doing that you must remove the alias you added in step (3), other wise the engine thinks the index already exists (because references to the alias translate into references to the new index).

CAUTION: My dashboards stopped working. I have an open ticket with Elastic's support at the moment waiting for instructions.

At this stage, after doing the above to all indexes that had warnings on date formats, you should reach the stage where you have no warnings left, and are ready for the upgrade.