Wednesday, April 24, 2019

Lazy, Filtered and Sorted Vaadin Grid using External Data Source

In large enterprise application development projects, it's usual to code use cases that, as the main requirement, define to show the user huge amounts of data in a tabular way. For this kind of scenarios, there is a particular web component that comes really handy: a Grid. Given that we like Vaadin's framework, we are going to describe how to configure it in such a way that doesn't restrict you about performance issues when doing that. 
This post assumes that you have some knowledge of web application development using the Vaadin Framework and the Java Platform. But we’ll try to explain everything as much as possible. The version that we used to write this application is Vaadin 13, but this part of the API is almost the same since Vaadin 8, so it should be possible to use the same approach in older versions.
Setting up a Vaadin Grid, is really easy, just create an instance and then customize the columns, or even skip that part if your Pojo is sufficiently self-descriptive.
As an example, let’s assume that we want to show a grid with data for a person, which we can define as a simplified entity like this:

public class Person {



 private String name;

 private String lastName;

 private Integer age;

 

 public Person(String name, String lastName, Integer age) {

  super();

  this.name = name;

  this.lastName = lastName;

  this.age = age;

 }

 public String getName() {

  return name;

 }

 public void setName(String name) {

  this.name = name;

 }

 public String getLastName() {

  return lastName;

 }

 public void setLastName(String lastName) {

  this.lastName = lastName;

 }

 public Integer getAge() {

  return age;

 }

 public void setAge(Integer age) {

  this.age = age;

 }

}

And then we have a service that will have the responsibility of retrieving this information from a backend, like a database or a remote API. Let’s call it PersonService.
Suppose we want to show those entities in a Vaadin Grid, we would do this:


     List<Person> persons = getPersonService().fetchPersons();

     Grid<Person> personsGrid = new Grid<>(Person.class);

     personsGrid.setItems(persons);

This is fine for small amounts of data, but when dealing with large data sets some features, such as lazy loading, filtering and sorting, must be left to the backend, implementing that all at once can be a complex task, so let's review them.

Lazy Loading

First let’s define what we are talking about when we use this term. Usually when we are handling a list of items, we can choose to obtain  all of them, or just the ones that we want to display or handle in some way. Obviously it depends on the amount. If we are obtaining one hundred of them, it is something that we can manage directly in memory, but if we are handling millions, we have to deal with them in a different way.
This is the case when a mechanism such as lazy loading makes things easier.
Vaadin components, like Grid or ComboBox, already support this kind of behavior, without the need of doing something special, but it only will take care of retrieving what is needed in a lazy loading fashion between the web browser and your web application.
Given that Vaadin only takes care of the presentation layer of your application, you need to do something to apply this behavior from your underlying data source (database, REST API, etc.).


According to the official documentation, you need to ask to your backend two questions:
Can you tell me the total amount of items that we are dealing with?
Of those, can you only give me the first N items, skipping the previous M items?
You don’t need to calculate N and M, those numbers are provided by the framework. They are basically the limit (N) and offset (M). You answer those two questions by providing two lambda expressions to the method DataProvider.fromCallbacks(), something like this:


DataProvider<Person, Void> dataProvider = DataProvider.fromCallbacks(

        // First callback fetches items based on a query

        query -> {

            // The index of the first item to load

            int offset = query.getOffset();

            // The number of items to load

            int limit = query.getLimit();

            List<Person> persons = getPersonService()

                    .fetchPersons(offset, limit);

            return persons.stream();

        },

        // Second callback fetches the number of items for a query

        query -> getPersonService().getPersonCount());

);

Grid<Person> grid = new Grid<>();

grid.setDataProvider(dataProvider);

Now we added two new different methods to our service class.
But let’s continue with another relevant requirement.

Filtering

It’s a rather common request to narrow down the size of the queried items, and one typical way of doing that is to make grids filterable.
But before digging into this, we need to define what is a Filter. For us it’s just a class that will contain values that we want to be taken into account when deciding if a given item will be returned or not.
In our case, let’s assume that we only want to filter people by their name and last name:


public class PersonFilter {

 

 private String nameFilter = null;

 private String lastNameFilter = null;



 public PersonFilter() { 

 }

 

 public PersonFilter(String nameFilter, String lastNameFilter) {

  this.setNameFilter(nameFilter);

  this.setLastNameFilter(lastNameFilter);

 }



 public String getNameFilter() {

  return nameFilter;

 }



 public String getLastNameFilter() {

  return lastNameFilter;

 }



 public void setNameFilter(String nameFilter) {

  this.nameFilter = nameFilter;

 }



 public void setLastNameFilter(String lastNameFilter) {

  this.lastNameFilter = lastNameFilter;

 }

}

Now that we have our filter, how to do the actual filtering?
Instead of calling DataProvider.fromCallbacks(), we are going to use DataProvider.fromFilteringCallbacks(), makes sense, right?


        DataProvider<Person, PersonFilter> dataProvider =

            DataProvider.fromFilteringCallbacks(

            query -> {

              Optional<PersonFilter> filter = query.getFilter();

              return getPersonService().fetchPersons(

                query.getOffset(),

                query.getLimit(),

                filter.map(f -> f.getNameFilter()).orElse(null),

                filter.map(f -> f.getLastNameFilter()).orElse(null)

              );

            },

            query -> {

              Optional<PersonFilter> filter = query.getFilter();

              return getPersonService().getPersonCount(

                filter.map(f -> f.getNameFilter()).orElse(null),

                filter.map(f -> f.getLastNameFilter()).orElse(null)

              );

            }

          );

In this case, what we are doing, is just to obtain the filter from the query, and send the filter values to the backend class. We cannot do the filtering by ourselves because if the underlying backend is a database, then those filters have to be taken into account when building the SQL queries. If we do the filtering by ourselves, then it’s impossible to calculate the final count of the items, without retrieving all of the entities from the backend, causing our lazy loading mechanism to fail its goal (not to retrieve everything). The same applies when calling a remote API.
But let’s continue, if we want to change the filters dynamically, then we need to be able to configure the filters, for that we need to call the method withConfigurableFilter(), that will return an enhanced DataProvider with a method that will allow us to set a given Filter instance:


        PersonFilter gridFilter = new PersonFilter();

        ConfigurableFilterDataProvider<Person,Void,PersonFilter> dp = dataProvider.withConfigurableFilter();

        dp.setFilter(gridFilter);

We can change the filter later, or just save a reference to this instance and modify it’s values. After doing that a call to the method refreshAll() in the DataProvider will trigger the loading of the data with the new filter values.
Now, what about the visual part? … let’s see:

        HeaderRow hr = personsGrid.prependHeaderRow();

        TextField nameFilterTF = new TextField();

        nameFilterTF.addValueChangeListener(ev->{

         gridFilter.setNameFilter(ev.getValue());

         dp.refreshAll();

        });

        hr.getCell(personsGrid.getColumnByKey("name")).setComponent(nameFilterTF);

        TextField lastNameFilterTF = new TextField();

        lastNameFilterTF.addValueChangeListener(ev->{

         gridFilter.setLastNameFilter(ev.getValue());

         dp.refreshAll();

        });

        hr.getCell(personsGrid.getColumnByKey("lastName")).setComponent(lastNameFilterTF);     

First we are creating a HeaderRow, that is a row that will show at the top of our fancy grid, that will contain components that will filter our data.
Then we are creating a TextField for holding the strings to filter the name and last name of our people.
Finally we are adding some value change listeners that will actually apply the filters when changing the value.
That’s great! … now we are loading filtered data in a lazy way using multiple filters.
But what if we need to sort the data? … let’s continue.

Sorting

Similarly to our filtering case, we need a class to store each sorting information that we need:


public class PersonSort {



 private String propertyName;

 private boolean descending;

 

 public String getPropertyName() {

  return propertyName;

 }

 public void setPropertyName(String propertyName) {

  this.propertyName = propertyName;

 }

 public boolean isDescending() {

  return descending;

 }

 public void setDescending(boolean descending) {

  this.descending = descending;

 }

  

}

The class is pretty self-explained. Just holding the property that is currently being sorted, and a boolean for specifying if we sort up or down.
You might think why we are constructing these classes (PersonFilter and PersonSort), given that we could just pass the values directly to the service. The main reason is to have framework agnostic classes to hold these kind of information and then pass them to the backend, making a cleaner separation of layers.
In our example, we are going to provide a new parameter to our PersonService: a list of PersonSort (sort orders). This is a list, because the order of the sort criteria is important, you can establish this order in the Grid component, by calling personsGrid.setMultiSort(true). Let’s review the changes in our code:


        DataProvider<Person, PersonFilter> dataProvider =

          DataProvider.fromFilteringCallbacks(

          query -> {

            Optional<PersonFilter> filter = query.getFilter();

            List<PersonSort> sortOrders = query.getSortOrders().stream().map(sortOrder->new PersonSort(sortOrder.getSorted(),sortOrder.getDirection().equals(SortDirection.ASCENDING))).collect(Collectors.toList());

            return getPersonService().fetchPersons(

              query.getOffset(),

              query.getLimit(),

              filter.map(f -> f.getNameFilter()).orElse(null),

              filter.map(f -> f.getLastNameFilter()).orElse(null),

              sortOrders

            );

          },

          query -> {

            Optional<PersonFilter> filter = query.getFilter();

            return getPersonService().getPersonCount(

              filter.map(f -> f.getNameFilter()).orElse(null),

              filter.map(f -> f.getLastNameFilter()).orElse(null)

            );

          }

        );

Now we are assembling this list of PersonSort, using query.getSortOrders().
The implementation of PersonService is irrelevant, because it's up to the backend to provide the data requested using the information provided:

  • Lazy loading information: offset and limit
  • Filtering information: name and last name filters
  • Sorting information: list of PersonSort

If the backend obtains the data from a relational database, then a SQL clause has to be created dynamically based on that information. For example:


SELECT *

FROM PERSONS P

WHERE P.NAME LIKE %FILTERNAME% AND P.LASTNAME LIKE %LASTNAME%

ORDER BY P.NAME DESC, P.LASTNAME ASC

LIMIT OFFSET,LIMIT

Here's a small animation showing our grid in action:


You can play around with this example, by checking this GitHub project with the sources.
Have fun!

Wednesday, April 3, 2019

Our 2nd year in review: keep flowing!

Another round around the sun, another term ended, and here we are still moving on. A lot of things happened since this team was created, and many of them last year. Here’s a quick review of what we did:

Traveling

Let’s start with one of the best experiences we got the chance to share as a team. We had the opportunity to visit the northern hemisphere to meet our friends at Vaadin’s main HQ.
While we were there, we shoot some awesome pictures, visit some interesting places and had the amazing chance of getting to know all of the experts behind the company that is responsible for this great open source framework.

Sharing

Talking about Open Source, we continued to support our community, and created a special section in our web site to group all of the projects that we maintain.
We worked hard to develop a bunch of interesting projects, giving away our best effort to make them production ready, so anyone can use our technologies in their projects, and participate with us actively so they can improve over time.
One interesting project to mention, was a small PWA application that displayed the results of the Football Worldcup in real time, showing how to integrate several technologies, so similar projects can be built using this a kickstart.

Learning

Delivering intelligence is not an easy task if you don’t find a way to expand your knowledge. We take this statement seriously: we schedule one learning day each week, so we can watch, learn and discuss about many technologies, methodologies, frameworks, languages, etc.
Here’s a list of the subjects and some interesting links for you to watch if you’re curious:


Writing

Having the opportunity to learn, and code, one final step is to share some of this knowledge, so we wrote some interesting articles about specific subjects, here’s a short list:


Speaking

But that is not all. We organized a talk in one of the most important universities in the area: Universidad Nacional del Litoral.
The chosen topic was to explain what is all the fuzz about Vaadin 10 and Web components.
You can find more information about it in our blog.

Delivering

We finished some interesting projects for new customers, growing our portfolio. Here is a summary of them, but with the promise that we will expand these experiences later in our site:

  • We continue working as consultants for big medical software implementations in U.S., and also giving technical and architectural support for big companies all around the globe.
  • Full SCM implementation for a customer in Colombia, that involved the installation and configuration of several servers, testing the entire stack, training several areas and finally taking care of every detail while going to production
  • Development support for a big application related to job openings, of a customer from Buenos Aires, that involved working with email marketing campaign tools and invocation of REST-APIs
  • Vaadin 7 to 13 migration, of a web module of an ERP software of a company from Rosario - Argentina

Networking

We continued sharing everything in our social networks, now you can find us in:


So that’s it, we are really happy with our results, so let’s finish with this quote from Albert Einstein:

Life is like riding a bicycle. To keep your balance you must keep moving.

In our case we will keep flowing. Thanks for reading!

Interested for our works and services?
Get more of our update !