Practical Data Manipulation Techniques in Java

Hello and welcome! Today, we're exploring practical data manipulation techniques in Java . We'll use Java lists to represent our data stream and perform projection, filtering, and aggregation. And here's the star of the show: our operations will be neatly packaged within a Java class! No mess, all clean code.

Introduction to Data Manipulation

Data manipulation is akin to being a sculptor but for data. We chisel and shape our data to get the desired structure. Java lists are perfect for this, and our operations will be conveniently bundled inside a Java class . So, let's get our toolbox ready! Here's a simple Java class, DataStream, that will serve as our toolbox: Java import java.util.List; public class DataStream<T> {private List<T> data; public DataStream(List<T> data) {this.data = data;} public List<T> getData() {return data;}}

Data Projection in Practice

Our first stop is data projection. Think of it like capturing a photo of our desired features. Suppose we have data about people. If we're only interested in names and ages, we project our data to include just these details. We'll extend our DataStream class with a projectData method for this: Java import java.util.List; import java.util.stream.Collectors; import java.util.Map; import java.util.HashMap; public class DataStream<T> {private List<T> data; public DataStream(List<T> data) {this.data = data;} public List<Map<String, Object>> projectData(List<String> keys) {return data.stream() .map(d -> {Map<String, Object> projectedMap = new HashMap<>(); for (String key : keys) {try {projectedMap.put(key, d.getClass().getDeclaredMethod("get" + key).invoke(d));} catch (Exception e) {e.printStackTrace();}} return projectedMap;}) .collect(Collectors.toList());}} class Person {private String name; private int age; private String profession; // Constructor and getters public Person(String name, int age, String profession) {this.name = name; this.age = age; this.profession = profession;} public String getName() { return name; } public int getAge() { return age; } public String getProfession() { return profession; }} public class Main {public static void main(String[] args) {List<Person> people = List.of(new Person("Alice", 25, "Engineer"), new Person("Bob", 30, "Doctor")); DataStream<Person> ds = new DataStream<>(people); List<Map<String, Object>> projectedData = ds.projectData(List.of("Name", "Age")); for (Map<String, Object> item : projectedData) {System.out.println(item.get("Name") + ", " + item.get("Age"));} // Outputs: // Alice, 25 // Bob, 30}} As you can see, we now have a new list with just the names and ages!

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Data Filtering in Practice

Next, we have data filtering, which is like cherry-picking our preferred data entries. We'll extend our DataStream class with a filterData method that uses a "test" function to filter data:

Java

import java.util.function.Predicate;
import java.util.stream.Collectors;

public class DataStream<T> {
    // Existing code...

    public DataStream<T> filterData(Predicate<T> predicate) {
        List<T> filteredData = data.stream()
                .filter(predicate)
                .collect(Collectors.toList());
        return new DataStream<>(filteredData);
    }
}

public class Main {
    public static void main(String[] args) {
        List<Person> people = List.of(
            new Person("Alice", 25, "Engineer"),
            new Person("Bob", 30, "Doctor")
        );

        DataStream<Person> ds = new DataStream<>(people);
        DataStream<Person> filteredDs = ds.filterData(person -> person.getAge() > 26);

        for (Person item : filteredDs.getData()) {
            System.out.println(item.getName() + ", " + item.getAge() + ", " + item.getProfession());
        }
        // Outputs:
        // Bob, 30, Doctor
    }
}

import java.util.function.Predicate;
import java.util.stream.Collectors;

public class DataStream<T> {
    // Existing code...

    public DataStream<T> filterData(Predicate<T> predicate) {
        List<T> filteredData = data.stream()
                .filter(predicate)
                .collect(Collectors.toList());
        return new DataStream<>(filteredData);
    }
}

public class Main {
    public static void main(String[] args) {
        List<Person> people = List.of(
            new Person("Alice", 25, "Engineer"),
            new Person("Bob", 30, "Doctor")
        );

        DataStream<Person> ds = new DataStream<>(people);
        DataStream<Person> filteredDs = ds.filterData(person -> person.getAge() > 26);

        for (Person item : filteredDs.getData()) {
            System.out.println(item.getName() + ", " + item.getAge() + ", " + item.getProfession());
        }
        // Outputs:
        // Bob, 30, Doctor
    }
}

With the filter method, our output is a list with only Bob’s data, as he's the only one who passes the "age over 26" test.

Data Aggregation in Practice

Last is data aggregation, where we condense our data into a summary. We will add an aggregateData method to our DataStream class for this:

Java

import java.util.function.Function;

public class DataStream<T> {
    // Existing code...

    public double aggregateData(Function<List<Double>, Double> aggregationFunction, 
                                String key) {
        List<Double> values = data.stream()
                .map(d -> {
                    try {
                        return (Double) d.getClass().getDeclaredMethod("get" + key).invoke(d);
                    } catch (Exception e) {
                        e.printStackTrace();
                        return 0.0;
                    }
                })
                .collect(Collectors.toList());

        return aggregationFunction.apply(values);
    }
}

public class Main {
    public static void main(String[] args) {
        List<Person> people = List.of(
            new Person("Alice", 25, "Engineer"),
            new Person("Bob", 30, "Doctor")
        );

        DataStream<Person> ds = new DataStream<>(people);
        double averageAge = ds.aggregateData(
            values -> values.stream().mapToDouble(v -> v).average().orElse(0), 
            "Age"
        );

        System.out.println(averageAge);  // Outputs: 27.5
    }
}

import java.util.function.Function;

public class DataStream<T> {
    // Existing code...

    public double aggregateData(Function<List<Double>, Double> aggregationFunction, 
                                String key) {
        List<Double> values = data.stream()
                .map(d -> {
                    try {
                        return (Double) d.getClass().getDeclaredMethod("get" + key).invoke(d);
                    } catch (Exception e) {
                        e.printStackTrace();
                        return 0.0;
                    }
                })
                .collect(Collectors.toList());

        return aggregationFunction.apply(values);
    }
}

public class Main {
    public static void main(String[] args) {
        List<Person> people = List.of(
            new Person("Alice", 25, "Engineer"),
            new Person("Bob", 30, "Doctor")
        );

        DataStream<Person> ds = new DataStream<>(people);
        double averageAge = ds.aggregateData(
            values -> values.stream().mapToDouble(v -> v).average().orElse(0), 
            "Age"
        );

        System.out.println(averageAge);  // Outputs: 27.5
    }
}

With this script, we calculate the average age, which is 27.5.

Combining Projection, Filtering, and Aggregation

Now, let's combine projection, filtering, and aggregation to see the collective power of these techniques. We'll extend our example to demonstrate this flow:

Data Projection: Choose only the desired fields.
Data Filtering: Filter the data based on certain conditions.
Data Aggregation: Summarize the filtered data.

We'll modify our DataStream class to include all the methods and then use them together in a workflow. On top of that, projection and filtering methods will now return an instance of DataStream<T>, not a list as before, so that we can chain these methods when calling them:

Java

import java.util.List;
import java.util.stream.Collectors;
import java.util.Map;
import java.util.HashMap;
import java.util.function.Predicate;
import java.util.function.Function;

public class DataStream<T> {
    private List<T> data;

    public DataStream(List<T> data) {
        this.data = data;
    }

    public DataStream<Map<String, Object>> projectData(List<String> keys) {
        List<Map<String, Object>> projectedData = data.stream()
                .map(d -> {
                    Map<String, Object> projectedMap = new HashMap<>();
                    for (String key : keys) {
                        try {
                            projectedMap.put(key, d.getClass().getDeclaredMethod("get" + key).invoke(d));
                        } catch (Exception e) {
                            e.printStackTrace();
                        }
                    }
                    return projectedMap;
                })
                .collect(Collectors.toList());

        return new DataStream<>(projectedData);
    }

    public DataStream<T> filterData(Predicate<T> predicate) {
        List<T> filteredData = data.stream()
                .filter(predicate)
                .collect(Collectors.toList());
        return new DataStream<>(filteredData);
    }

    public double aggregateData(Function<List<Double>, Double> aggregationFunction, 
                                String key) {
        List<Double> values = data.stream()
                .map(d -> {
                    try {
                        return (Double) d.getClass().getDeclaredMethod("get" + key).invoke(d);
                    } catch (Exception e) {
                        e.printStackTrace();
                        return 0.0;
                    }
                })
                .collect(Collectors.toList());

        return aggregationFunction.apply(values);
    }
}

class Person {
    private String name;
    private int age;
    private String profession;
    private double salary;

    // Constructor and getters
    public Person(String name, int age, String profession, double salary) {
        this.name = name;
        this.age = age;
        this.profession = profession;
        this.salary = salary;
    }

    public String getName() { return name; }
    public int getAge() { return age; }
    public double getSalary() { return salary; }
    public String getProfession() { return profession; }
}

public class Main {
    public static void main(String[] args) {
        List<Person> people = List.of(
            new Person("Alice", 25, "Engineer", 70000),
            new Person("Bob", 30, "Doctor", 120000),
            new Person("Carol", 35, "Artist", 50000),
            new Person("David", 40, "Engineer", 90000)
        );

        // Step 1: Project the data to include only 'Name', 'Age', and 'Salary'
        DataStream<Map<String, Object>> projectedDs = new DataStream<>(people)
            .projectData(List.of("Name", "Age", "Salary"));

        // Step 2: Filter the projected data to include only those with age > 30
        DataStream<Map<String, Object>> filteredDs = projectedDs.filterData(
            map -> (int) map.get("Age") > 30
        );

        // Step 3: Aggregate the filtered data to compute the average salary
        double averageSalary = filteredDs.aggregateData(
            salaries -> salaries.stream().mapToDouble(v -> v).average().orElse(0),
            "Salary"
        );

        System.out.println(averageSalary);  // Outputs: 70000.0
    }
}

import java.util.List;
import java.util.stream.Collectors;
import java.util.Map;
import java.util.HashMap;
import java.util.function.Predicate;
import java.util.function.Function;

public class DataStream<T> {
    private List<T> data;

    public DataStream(List<T> data) {
        this.data = data;
    }

    public DataStream<Map<String, Object>> projectData(List<String> keys) {
        List<Map<String, Object>> projectedData = data.stream()
                .map(d -> {
                    Map<String, Object> projectedMap = new HashMap<>();
                    for (String key : keys) {
                        try {
                            projectedMap.put(key, d.getClass().getDeclaredMethod("get" + key).invoke(d));
                        } catch (Exception e) {
                            e.printStackTrace();
                        }
                    }
                    return projectedMap;
                })
                .collect(Collectors.toList());

        return new DataStream<>(projectedData);
    }

    public DataStream<T> filterData(Predicate<T> predicate) {
        List<T> filteredData = data.stream()
                .filter(predicate)
                .collect(Collectors.toList());
        return new DataStream<>(filteredData);
    }

    public double aggregateData(Function<List<Double>, Double> aggregationFunction, 
                                String key) {
        List<Double> values = data.stream()
                .map(d -> {
                    try {
                        return (Double) d.getClass().getDeclaredMethod("get" + key).invoke(d);
                    } catch (Exception e) {
                        e.printStackTrace();
                        return 0.0;
                    }
                })
                .collect(Collectors.toList());

        return aggregationFunction.apply(values);
    }
}

class Person {
    private String name;
    private int age;
    private String profession;
    private double salary;

    // Constructor and getters
    public Person(String name, int age, String profession, double salary) {
        this.name = name;
        this.age = age;
        this.profession = profession;
        this.salary = salary;
    }

    public String getName() { return name; }
    public int getAge() { return age; }
    public double getSalary() { return salary; }
    public String getProfession() { return profession; }
}

public class Main {
    public static void main(String[] args) {
        List<Person> people = List.of(
            new Person("Alice", 25, "Engineer", 70000),
            new Person("Bob", 30, "Doctor", 120000),
            new Person("Carol", 35, "Artist", 50000),
            new Person("David", 40, "Engineer", 90000)
        );

        // Step 1: Project the data to include only 'Name', 'Age', and 'Salary'
        DataStream<Map<String, Object>> projectedDs = new DataStream<>(people)
            .projectData(List.of("Name", "Age", "Salary"));

        // Step 2: Filter the projected data to include only those with age > 30
        DataStream<Map<String, Object>> filteredDs = projectedDs.filterData(
            map -> (int) map.get("Age") > 30
        );

        // Step 3: Aggregate the filtered data to compute the average salary
        double averageSalary = filteredDs.aggregateData(
            salaries -> salaries.stream().mapToDouble(v -> v).average().orElse(0),
            "Salary"
        );

        System.out.println(averageSalary);  // Outputs: 70000.0
    }
}

Here:

Projection: We choose only the Name, Age, and Salary fields from our data. The projectData method now returns a DataStream<Map<String, Object>> object, allowing us to chain multiple operations.
Filtering: We filter the projected data to include only those persons whose age is greater than 30. The filterData method also returns a DataStream<T> object for chaining.
Aggregation: We calculate the average salary of the filtered data. The final output shows the average salary for those aged over 30, which is 70,000.

By combining these methods, our data manipulation becomes both powerful and concise. Try experimenting and see what you can create!

Brilliant job! You've now grasped the basics of data projection, filtering, and aggregation on Java lists. Plus, you've learned to package these operations in a Java class — a neat bundle of reusable code magic!

Now, why not try applying these fresh skills with some practice exercises? They're just around the corner. Ready? Let's dive into more fun with data manipulation!

Practical Data Manipulation Techniques in Java