Practical Data Manipulation Techniques with Kotlin

Topic Overview and Importance

Hello and welcome! Today, we're exploring practical data manipulation techniques in Kotlin . We'll use Kotlin List collections to represent our data stream and perform projection, filtering, and aggregation. And here's the star of the show: our operations will be neatly packaged within a Kotlin class! No mess, all clean code.

Introduction to Data Manipulation

Data manipulation is akin to being a sculptor but for data. We chisel and shape our data to get the desired structure. Kotlin lists are perfect for this, and our operations will be conveniently bundled inside a Kotlin class. So, let's get our toolbox ready! Here's a simple Kotlin class, DataStream, that will serve as our toolbox: Kotlin class DataStream(private val data: List<Map<String, Any?>>)

Data Projection in Practice

Our first stop is data projection. Think of it as capturing a photo of our desired features. Suppose we have data about people, and we're only interested in their names and ages; we project our data to include just these details. We'll extend our DataStream class with a projectData method for this: Kotlin class DataStream(private val data: List<Map<String, Any?>>) {fun projectData(keys: List<String>): DataStream {val projectedData = data.map { entry -> entry.filterKeys { it in keys }} return DataStream(projectedData)} fun dataToString(): String {return data.joinToString(separator = ", ", prefix = "[", postfix = "]") {it.entries.joinToString(separator = ", ", prefix = "{", postfix = "}") { entry -> "${entry.key}=${entry.value}"}}}} fun main() {// Let's use it! val ds = DataStream(listOf(mapOf("name" to "Alice", "age" to 25, "profession" to "Engineer"), mapOf("name" to "Bob", "age" to 30, "profession" to "Doctor"))) val projectedDs = ds.projectData(listOf("name", "age")) println(projectedDs.dataToString()) // Outputs: [{name=Alice, age=25}, {name=Bob, age=30}]} As you can see, we now have a new list with just the names and ages!

Data Filtering in Practice

Next, we have data filtering, which is like cherry-picking our preferred data entries. We'll extend our DataStream class with a filterData method that uses a "predicate" function to filter data: Kotlin class DataStream(private val data: List<Map<String, Any?>>) {// ... other methods ... fun filterData(predicate: (Map<String, Any?>) -> Boolean): DataStream {val filteredData = data.filter(predicate) return DataStream(filteredData)}} fun main() {// Applying it: val ds = DataStream(listOf(mapOf("name" to "Alice", "age" to 25, "profession" to "Engineer"), mapOf("name" to "Bob", "age" to 30, "profession" to "Doctor"))) val ageTest: (Map<String, Any?>) -> Boolean = { it["age"] as Int > 26 } val filteredDs = ds.filterData(ageTest) println(filteredDs.dataToString()) // Outputs: [{name=Bob, age=30, profession=Doctor}]} With the filter method, our output is a list with only Bob’s data, as he's the only one who passes the "age over 26" test.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Combining Projection, Filtering, and Aggregation

Now, let's combine projection, filtering, and aggregation to see the collective power of these techniques. We'll extend our example to demonstrate this flow:

Data Projection: Choose only the desired fields.
Data Filtering: Filter the data based on certain conditions.
Data Aggregation: Summarize the filtered data.

We'll modify our DataStream class to include all the methods and then use them together in a workflow. The projection and filtering methods will now return an instance of DataStream, not a list as before, so that we can chain these methods when calling them:

Kotlin

class DataStream(private val data: List<Map<String, Any?>>) {

    fun projectData(keys: List<String>): DataStream {
        val projectedData = data.map { entry ->
            entry.filterKeys { it in keys }
        }
        return DataStream(projectedData)
    }

    fun filterData(predicate: (Map<String, Any?>) -> Boolean): DataStream {
        val filteredData = data.filter(predicate)
        return DataStream(filteredData)
    }

    fun aggregateData(key: String, aggFunc: (List<Int>) -> Double): Double {
        val values = data.mapNotNull { it[key] as? Int }
        return aggFunc(values)
    }
}

fun main() {
    // Example usage
    val ds = DataStream(
        listOf(
            mapOf("name" to "Alice", "age" to 25, "profession" to "Engineer", "salary" to 70000),
            mapOf("name" to "Bob", "age" to 30, "profession" to "Doctor", "salary" to 120000),
            mapOf("name" to "Carol", "age" to 35, "profession" to "Artist", "salary" to 50000),
            mapOf("name" to "David", "age" to 40, "profession" to "Engineer", "salary" to 90000)
        )
    )

    // Step 1: Project the data to include only 'name', 'age', and 'salary'
    val projectedDs = ds.projectData(listOf("name", "age", "salary"))

    // Step 2: Filter the projected data to include only those with age > 30
    val filteredDs = projectedDs.filterData { it["age"] as Int > 30 }

    // Step 3: Aggregate the filtered data to compute the average salary
    val averageSalary = filteredDs.aggregateData("salary") { salaries -> salaries.average() }
    println(averageSalary)  // Outputs: 70000.0
}

class DataStream(private val data: List<Map<String, Any?>>) {

    fun projectData(keys: List<String>): DataStream {
        val projectedData = data.map { entry ->
            entry.filterKeys { it in keys }
        }
        return DataStream(projectedData)
    }

    fun filterData(predicate: (Map<String, Any?>) -> Boolean): DataStream {
        val filteredData = data.filter(predicate)
        return DataStream(filteredData)
    }

    fun aggregateData(key: String, aggFunc: (List<Int>) -> Double): Double {
        val values = data.mapNotNull { it[key] as? Int }
        return aggFunc(values)
    }
}

fun main() {
    // Example usage
    val ds = DataStream(
        listOf(
            mapOf("name" to "Alice", "age" to 25, "profession" to "Engineer", "salary" to 70000),
            mapOf("name" to "Bob", "age" to 30, "profession" to "Doctor", "salary" to 120000),
            mapOf("name" to "Carol", "age" to 35, "profession" to "Artist", "salary" to 50000),
            mapOf("name" to "David", "age" to 40, "profession" to "Engineer", "salary" to 90000)
        )
    )

    // Step 1: Project the data to include only 'name', 'age', and 'salary'
    val projectedDs = ds.projectData(listOf("name", "age", "salary"))

    // Step 2: Filter the projected data to include only those with age > 30
    val filteredDs = projectedDs.filterData { it["age"] as Int > 30 }

    // Step 3: Aggregate the filtered data to compute the average salary
    val averageSalary = filteredDs.aggregateData("salary") { salaries -> salaries.average() }
    println(averageSalary)  // Outputs: 70000.0
}

Here:

Projection: We choose only the name, age, and salary fields from our data. The projectData method now returns a DataStream object, allowing us to chain multiple operations.
Filtering: We filter the projected data to include only those persons whose age is greater than 30. The filterData method also returns a DataStream object for chaining.
Aggregation: We calculate the average salary of the filtered data. The final output shows the average salary for those aged over 30, which is 70,000.

By combining these methods, our data manipulation becomes both powerful and concise. Try experimenting and see what you can create!