Pagination

In general, a pagination system allows to avoid returning too much data to the user. It also allows to know how much data there is in total, how many pages there are, ... It is a particularly useful tool when working with api. Unfortunately, this system does not apply to all cases. For example, in the case of reading a single element, it is not necessary to add a pagination. However, to list a lot of data, the pagination becomes important

Here we will create our own pagination system based on the Doctrine one.

Introducing the Paging System

For this lesson, we will create a simple but effective system. From only 2 parameters, our paging system will retrieve the following information:

• the total
• the number of data on the current page
• the total number of pages
• the current page
• if there is a next page
• if there is a previous page
• ...

In idea, we will simply provide the desired page and the request coming from Doctrine to our pagination object and, on its side, it will retrieve the information we have quoted.

Pagination structure

In order to reinvent the wheel as little as possible, we will base our pagination system on the Doctrine one. In a folder called src/Model, we will create a new file called Paginator.php.

Here is the code for our Paginator object.

// src/Model/Paginator.php

namespace App\Model;

use Doctrine\ORM\Query;
use Doctrine\ORM\QueryBuilder;
use Doctrine\ORM\Tools\Pagination\Paginator as DoctrinePaginator;
use ArrayIterator;

class Paginator extends DoctrinePaginator
{
    public const ITEMS_PER_PAGE = 5;

    private int $total;
    private array $data;
    private int $count;
    private int $totalpages;
    private int $page;

    public function __construct(QueryBuilder|Query $query, int $page = 1, bool $fetchJoinCollection = true)
    {
        $query->setFirstResult(($page - 1) * self::ITEMS_PER_PAGE);
        $query->setMaxResults(self::ITEMS_PER_PAGE);

        parent::__construct($query, $fetchJoinCollection);
        $this->total = $this->count();
        $this->data = iterator_to_array(parent::getIterator());
        $this->count = count($this->data);
        $this->page = $page;

        try {
            $this->totalpages = ceil($this->total / self::ITEMS_PER_PAGE);
        } catch (\DivisionByZeroError $e) {
            $this->totalpages = 0;
        }
    }

    public function getTotal(): int
    {
        return $this->total;
    }

    public function getData(): array
    {
        return $this->data;
    }

    public function getCount(): int
    {
        return $this->count;
    }

    public function getTotalPages(): int
    {
        return $this->totalpages;
    }

    public function getCurrentPage(): int
    {
        return $this->page;
    }

    public function getItemsPerPage(): ?int
    {
        return $this->getQuery()->getMaxResults();
    }

    public function getOffset(): ?int
    {
        return $this->getQuery()->getFirstResult();
    }

    public function hasNextPage(): bool
    {
        if ($this->getCurrentPage() >= 1 && $this->getCurrentPage() < $this->getTotalPages()) {
            return true;
        }

        return false;
    }

    public function hasPreviousPage(): bool
    {
        if ($this->getCurrentPage() > 1 && $this->getCurrentPage() <= $this->getTotalPages()) {
            return true;
        }

        return false;
    }

    public function getIterator(): ArrayIterator
    {
        return new ArrayIterator([
            'data' => $this->getData(),
            'pagination' => [
                'total' => $this->getTotal(),
                'count' => $this->getCount(),
                'offset' => $this->getOffset(),
                'items_per_page' => $this->getItemsPerPage(),
                'total_pages' => $this->getTotalPages(),
                'current_page' => $this->getCurrentPage(),
                'has_next_page' => $this->hasNextPage(),
                'has_previous_page' => $this->hasPreviousPage(),
            ],
        ]);
    }
}

As we have mentioned, our Paginator object can be made with only 2 parameters:

• The request
• The desired page

Here, the desired page will be used to fill in the offset. This offset will be used in the SQL query to retrieve only the elements corresponding to the desired page.

Let's take the time to detail the different methods.

• The getTotal, getData, getCount, getTotalPages and getCurrentPage methods are simply getter for the total, data, count, totalPages and page attributes.
• The getItemsPerPage method is a utility method that returns the number of items per page. The value corresponds to the limit given in the SQL query. It is also the same value that is filled in the ITEMS_PER_PAGE constant.
• The getOffset method, like the previous one, is a utility method to retrieve the offset that is filled in the SQL query.
• The hasNextPage method allows to know if another page exists after the current page.
• Conversely, the method hasPreviousPage allows to know if a page existed before the current page.
• Finally, the most important part, the getIterator method allows to format the result of the pagination in the desired format. This result will be used when serializing the pagination object.

Going back to this getIterator method, we can notice that we define two main fields: data and pagination.

The data field simply lists the elements returned by the SQL query after pagination.

The pagination field, on the other hand, allows to return information about the pagination. It is here that we find the different methods we defined previously.

Now that we have our object allowing us to paginate a query, we just have to use it in our queries.

Implementation in existing code

The use of our Paginator is rather. We will mostly use it in repositories.

Remember that the pagination system will only be used in the case where a list of elements is returned. The pagination will not be used for modification, deletion or creation requests.

Let's take as support the /todos request with the GET method.

Currently, we have the following code:

// src/Controller/TodoController.php

#[Route('/todos', name: 'get_todos', methods: ["GET"])]
public function getTodos(TodoRepository $todoRepository): JsonResponse
{
    $todos = $todoRepository->findAll();

    return $this->json($todos);
}

Unfortunately, the result returned by the findAll method is a list of all elements without any restriction. We'll try to use our pagination system instead. To do this, we'll have to start by creating a new method in the Todo repository. This method will return a paged list to the controller. Let's call this new method findAllWithPagination. Since our Paginator object needs the desired page, we'll need to enter it into our repository.

// src/Repository/TodoRepository.php

use App\Model\Paginator;

public function findAllWithPagination(int $page): Paginator
{
    // ...
}

Then, we will simply have to create our query without filling in the limit or offset because they will be handled by the Paginator directly. In our case, we want to retrieve all the items sorted from oldest to newest.

// src/Repository/TodoRepository.php

$query = $this->createQueryBuilder('t')->orderBy('t.createdAt', 'DESC');

Then, we will just have to fill this request to our Paginator object with the desired page, and it will be done.

// src/Repository/TodoRepository.php

return new Paginator($query, $page);

You should end up with the following method.

// src/Repository/TodoRepository.php

public function findAllWithPagination(int $page): Paginator
{
    $query = $this->createQueryBuilder('t')->orderBy('t.createdAt', 'ASC');

    return new Paginator($query, $page);
}

Finally, in the controller, rather than calling the findAll method, we will instead call our new findAllWithPagination method.

// src/Controller/TodoController.php

#[Route('/todos', name: 'get_todos', methods: ["GET"])]
public function getTodos(TodoRepository $todoRepository): JsonResponse
{
    $todos = $todoRepository->findAllWithPagination(1);

    return $this->json($todos);
}

Now, if we go to /api/todos with the GET method you should get the following result.

{
    "data": [
        {
            "id": 1,
            "title": "Updated title",
            "createdAt": "2023-04-23T12:38:53+00:00",
            "updatedAt": "2023-04-23T13:29:52+00:00",
            "completed": false
        },
        {
            "id": 2,
            "title": "Updated title",
            "createdAt": "2023-04-23T12:38:53+00:00",
            "updatedAt": "2023-04-23T13:28:45+00:00",
            "completed": false
        },
        {
            "id": 3,
            "title": "Et et et vero vel aut assumenda et. Voluptatem repudiandae accusantium dolor ad quae exercitationem voluptas. Voluptatem quis aspernatur sed ab laudantium sequi. Id omnis accusantium laudantium culpa vero in sunt.",
            "createdAt": "2023-04-23T12:38:53+00:00",
            "updatedAt": "2023-04-23T12:38:53+00:00",
            "completed": false
        },
        {
            "id": 4,
            "title": "Animi velit et aut consequuntur. Tempora impedit quidem nobis explicabo nobis doloribus qui. Cumque cumque commodi illum voluptatem necessitatibus quia sed doloremque.",
            "createdAt": "2023-04-23T12:38:53+00:00",
            "updatedAt": "2023-04-23T12:38:53+00:00",
            "completed": false
        },
        {
            "id": 5,
            "title": "Reiciendis delectus ut sed delectus. Nihil esse voluptatem qui inventore. Eos ipsam eveniet ut molestiae. Tempore nihil consequatur ut labore aliquid aliquam ut.",
            "createdAt": "2023-04-23T12:38:53+00:00",
            "updatedAt": "2023-04-23T12:38:53+00:00",
            "completed": false
        }
    ],
    "pagination": {
        "total": 6,
        "count": 5,
        "offset": 0,
        "items_per_page": 5,
        "total_pages": 2,
        "current_page": 1,
        "has_next_page": true,
        "has_previous_page": false
    }
}

Unfortunately, for the moment, this route only returns the todos of the first page. So we will have to manage the pages via the query query parameters and the page change.

Page management

In many cases, the management of pages and other parameters such as sorting or the number of elements desired, are managed via the query parameters. These are the parameters found in the url after a ?. For example the url /api/todos?page=2 contains the query parameter page which has the value 2.

As for the validation of the data sent by the user, we will have to validate the page parameter in order to avoid errors. To do this, we will reuse the Options Resolver component.

First, let's create our Options Resolver. In the file /src/OptionsResolver/PaginatorOptionsResolver.php, let us fill in the following content.

// src/OptionsResolver/PaginatorOptionsResolver.php

namespace App\OptionsResolver;

use Symfony\Component\OptionsResolver\Options;
use Symfony\Component\OptionsResolver\OptionsResolver;

class PaginatorOptionsResolver extends OptionsResolver
{
    public function configurePage(): self
    {
        return $this
            ->setDefined("page")
            ->setDefault("page", 1)
            ->setAllowedTypes("page", "numeric")
            ->setAllowedValues("page", function ($page) {
                $validatedValue = filter_var($page, FILTER_VALIDATE_INT, [
                    'flags' => FILTER_NULL_ON_FAILURE,
                ]);

                if(null === $validatedValue || $validatedValue < 1) {
                    return false;
                }

                return true;
            })
            ->setNormalizer("page", fn (Options $options, $page) => (int) $page);
    }
}

The principle is similar to the previous options resolver we were able to create. In the configurePage method, we specify the following:

• We indicate that the page field exists
• We set the default value of the page field
• We specify the allowed type. Although the page is an integer, we cannot use the int type because the variables from the query parameters are necessarily of type string. On the other hand, the numeric type allows us to determine if a string corresponds to a number.
• We define the allowed values. Here, we want the numeric value to be a positive integer
• Finally we create a normalizer in order to format our page field as an integer. Without this operation, the value of the page field will be of type string and not of type integer.

To use it, as before, we just need to inject our Options Resolver into the desired method via the method parameters and then use it.

As a reminder, we will only use paging for the /api/todos request with the GET HTTP method.

So we end up with the following code.

// src/Controller/TodoController.php

use App\OptionsResolver\PaginatorOptionsResolver;

#[Route('/todos', name: 'get_todos', methods: ["GET"])]
public function getTodos(TodoRepository $todoRepository, Request $request, PaginatorOptionsResolver $paginatorOptionsResolver): JsonResponse
{
    try {
        $queryParams = $paginatorOptionsResolver
            ->configurePage()
            ->resolve($request->query->all());

        $todos = $todoRepository->findAllWithPagination($queryParams["page"]);

        return $this->json($todos);
    } catch(Exception $e) {
        throw new BadRequestHttpException($e->getMessage());
    }
}

The principle is the same as before. We validate the query parameters from the $request->query->all() method. Then we use page in our repository.

Finally, we use a try catch so as to return a 400 Bad Request error instead of 500 Internal Server Error.

Now, if we try our /api/todos route again with the GET method, we end up with the same result as before. However, this time we can change the desired page by changing the page parameter in the url.

In summary

In this lesson we learned how to create a functional pagination from the Doctrine Paginator. We also reused the Options Resolver component in order to validate the page parameter that the user sends us. Finally, we discovered the :

• setDefault
• setAllowedValues
• setNormalizer

as part of the Options Resolvers.

The functional part of our api is now complete. In other words, the returned results will not change from now on. However, the course is not finished. Indeed, we still have to implement a security system to our api via the bearer token. This is the subject of the next lesson.