Now, let's combine projection, filtering, and aggregation to see the collective power of these techniques. We'll extend our example to demonstrate this flow:
- Data Projection: Choose only the desired fields.
- Data Filtering: Filter the data based on certain conditions.
- Data Aggregation: Summarize the filtered data.
We'll modify our DataStream class to include all the methods and then use them together in a workflow. On top of that, projection and filtering methods will now return an instance of DataStream<T>, not a list as before, so that we can chain these methods when calling them:
import java.util.List;
import java.util.stream.Collectors;
import java.util.Map;
import java.util.HashMap;
import java.util.function.Predicate;
import java.util.function.Function;
public class DataStream<T> {
private List<T> data;
public DataStream(List<T> data) {
this.data = data;
}
public DataStream<Map<String, Object>> projectData(List<String> keys) {
List<Map<String, Object>> projectedData = data.stream()
.map(d -> {
Map<String, Object> projectedMap = new HashMap<>();
for (String key : keys) {
try {
projectedMap.put(key, d.getClass().getDeclaredMethod("get" + key).invoke(d));
} catch (Exception e) {
e.printStackTrace();
}
}
return projectedMap;
})
.collect(Collectors.toList());
return new DataStream<>(projectedData);
}
public DataStream<T> filterData(Predicate<T> predicate) {
List<T> filteredData = data.stream()
.filter(predicate)
.collect(Collectors.toList());
return new DataStream<>(filteredData);
}
public double aggregateData(Function<List<Double>, Double> aggregationFunction,
String key) {
List<Double> values = data.stream()
.map(d -> {
try {
return (Double) d.getClass().getDeclaredMethod("get" + key).invoke(d);
} catch (Exception e) {
e.printStackTrace();
return 0.0;
}
})
.collect(Collectors.toList());
return aggregationFunction.apply(values);
}
}
class Person {
private String name;
private int age;
private String profession;
private double salary;
// Constructor and getters
public Person(String name, int age, String profession, double salary) {
this.name = name;
this.age = age;
this.profession = profession;
this.salary = salary;
}
public String getName() { return name; }
public int getAge() { return age; }
public double getSalary() { return salary; }
public String getProfession() { return profession; }
}
public class Main {
public static void main(String[] args) {
List<Person> people = List.of(
new Person("Alice", 25, "Engineer", 70000),
new Person("Bob", 30, "Doctor", 120000),
new Person("Carol", 35, "Artist", 50000),
new Person("David", 40, "Engineer", 90000)
);
// Step 1: Project the data to include only 'Name', 'Age', and 'Salary'
DataStream<Map<String, Object>> projectedDs = new DataStream<>(people)
.projectData(List.of("Name", "Age", "Salary"));
// Step 2: Filter the projected data to include only those with age > 30
DataStream<Map<String, Object>> filteredDs = projectedDs.filterData(
map -> (int) map.get("Age") > 30
);
// Step 3: Aggregate the filtered data to compute the average salary
double averageSalary = filteredDs.aggregateData(
salaries -> salaries.stream().mapToDouble(v -> v).average().orElse(0),
"Salary"
);
System.out.println(averageSalary); // Outputs: 70000.0
}
}
Here:
- Projection: We choose only the
Name, Age, and Salary fields from our data. The projectData method now returns a DataStream<Map<String, Object>> object, allowing us to chain multiple operations.
- Filtering: We filter the projected data to include only those persons whose age is greater than 30. The
filterData method also returns a DataStream<T> object for chaining.
- Aggregation: We calculate the average salary of the filtered data. The final output shows the average salary for those aged over 30, which is
70,000.
By combining these methods, our data manipulation becomes both powerful and concise. Try experimenting and see what you can create!