Search This Blog

Monday, December 9, 2019

Elasticsearch aggregation - show aggregations only + overcome default limitation of aggregated buckets

Elasticsearch aggregation - output showing only aggregated field + overcoming default limitation of aggregated values (in this case, set to 1000)

GET customer_success_reports/_search
{
  "size":0,
    "aggs" : {
        "Country": {
            "composite" : {
              "size": 1000,
                "sources" : [
                    { "country": { "terms" : { "field": "Meta.GeoInfo.Country.keyword","missing_bucket": true } } }
                ]
            }
        }
     }
}


size:0 -> makes the output show the aggregated buckets only instead of the whole documents

Sunday, November 3, 2019

Fixing date field formats to upgrade elastic to version 7

Here is what did, that eventually worked:


  1. The only way I found to EDIT the field format was by creating a new (empty) index, by specifying its mapping. The date fields I replaced (had my json in a text editor and did a "replace all" on each different format to a elastic-7-complying one. In my case, most of my fields were "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis", and I changed them to "8yyyy/MM/dd HH:mm:ss||8yyyy/MM/dd".
  2. Then I issued a _reindex, meaning I copied the contents of each index I had to fix to its corresponding new empty index I created, in the step above, to hold its data, now in a compatible format
  3. At the end of the reindex (do not worry about the "backend connection closed" message, the copy keeps running, check the "Reload Indices" in Index Management), I added an alias to the new index so that new documents being added to the original one are now inserted in the new one (using the _alias api)
  4. If you want the indexes to remain with their original names at the end, you can then copy once again the indexes to a new one, using the original name this time. Be aware that before doing that you must remove the alias you added in step (3), other wise the engine thinks the index already exists (because references to the alias translate into references to the new index).

CAUTION: My dashboards stopped working. I have an open ticket with Elastic's support at the moment waiting for instructions.

At this stage, after doing the above to all indexes that had warnings on date formats, you should reach the stage where you have no warnings left, and are ready for the upgrade.

Tuesday, September 24, 2019

Get Enum Attribute Value in C#

public static string GetEnumAttributeValue(Enum enumValue, Type attributeType, string attributePropertyName)
        {
            /*
             * Extracts a given attribute value from an enum:
             *
             * Ex:
             * public enum X
             * {
                     [MyAttribute(myProp = "aaaa")]
             *       x1,
             *       x2,
             *       [Description("desc")]
             *       x3
             * }
             *
             * Usage:
             *      GetEnumAttribute(X.x1, typeof(MyAttribute), "myProp") returns "aaaa"
             *      GetEnumAttribute(X.x2, typeof(MyAttribute), "myProp") returns string.Empty
             *      GetEnumAttribute(X.x3, typeof(DescriptionAttribute), "Description") returns "desc"
             */

            var attributeObj = enumValue.GetType()?.GetMember(enumValue.ToString())?.FirstOrDefault()?.GetCustomAttributes(attributeType, false)?.FirstOrDefault();

            if (attributeObj == null)
                return string.Empty;
            else
            {
                try
                {
                    var attributeCastedObj = Convert.ChangeType(attributeObj, attributeType);
                    var attributePropertyValue = attributeType.GetProperty(attributePropertyName)?.GetValue(attributeCastedObj);
                    return attributePropertyValue?.ToString() ?? string.Empty;
                }
                catch (Exception ex)
                {
                    return string.Empty;
                }
            }
        }