How to create smaller chunks of record collection from the large list of records in PowerApps?

Summary

The need was to create smaller chunks of records from the large list of record collections in the PowerApps using PowerFx functions.

To demonstrate the large list I have generated it using the Sequence PowerFx function. I used the “cur_date” and “ID” properties. The “ID” property is useful in the logic below, for your large list just simply create an ID property or anything else to put the record number value.

// just create 5,000 dates
// Make sure you have ID property with values as record number.
ClearCollect (
    LargeTestList,
    ForAll (
        Sequence(5000),
        {
            cur_date: DateAdd(
                Today(),
                Value,
                TimeUnit.Days
            ),
            ID: Value
        }
    )
);

Solution

Here is the solution code for the above large list to create chunks of 500 records.

// First get the first record of the large list
Set(
    firstRecord,
    First(LargeTestList)
);
// Now get the last record of the large list. NOTE: we make use of ID property.
Set(
    lastRecord,
    First(
        Sort(
            LargeTestList,
            ID,
            SortOrder.Descending
        )
    )
);
// Now, we need to check the large list count of rows how many chunks can be created.
Set(
    iterationsNo,
    RoundUp(
        CountRows(LargeTestList) / 500,
        0
    )
);
// Based on the number of chucks finding, now create the list of possible iterations
ClearCollect(
    iterations,
    Sequence(
        iterationsNo,
        0
    )
);
// Iterate over the iterations.
ForAll(
    iterations,
    With(
        {
            prevThreshold: Value(Value)       * 500,
            nextThreshold: (Value(Value) + 1) * 500
        },
        If (
            lastRecord.ID > Value,
            // Send the 500 resource to the FHIR
            // To similify the logic I have done collect here. But you can 
            // do whaterver with the chuncked data.
            Collect(
                datasource_temp,
                Filter(
                    LargeTestList,
                    ID > prevThreshold && ID <= nextThreshold
                )
            )
        )
    )
);

Conclusion

It took me some time to figure out how to create the chunks in Power Fx. I hope it may be useful to you if you have a similar need. To simplify the complexity of the Large collection I created a TEST collection here but you can simply add ID field in the collection.

About Pankaj

I am a Developer and my linked profile is https://www.linkedin.com/in/pankajsurti/
This entry was posted in FHIR, Power Apps. Bookmark the permalink.

Leave a comment