Pagination
In general, a pagination system allows to avoid returning too much data to the user. It also allows to know how much data there is in total, how many pages there are, ... It is a particularly useful tool when working with api. Unfortunately, this system does not apply to all cases. For example, in the case of reading a single element, it is not necessary to add a pagination. However, to list a lot of data, the pagination becomes important
Here we will create our own pagination system based on the Doctrine one.
Introducing the Paging System
For this lesson, we will create a simple but effective system. From only 2 parameters, our paging system will retrieve the following information:
- the total
- the number of data on the current page
- the total number of pages
- the current page
- if there is a next page
- if there is a previous page
- ...
In idea, we will simply provide the desired page and the request coming from Doctrine to our pagination object and, on its side, it will retrieve the information we have quoted.
Pagination structure
In order to reinvent the wheel as little as possible, we will base our pagination system on the Doctrine one. In a folder called src/Model
, we will create a new file called Paginator.php
.
Here is the code for our Paginator
object.
// src/Model/Paginator.php
namespace App\Model;
use Doctrine\ORM\Query;
use Doctrine\ORM\QueryBuilder;
use Doctrine\ORM\Tools\Pagination\Paginator as DoctrinePaginator;
use ArrayIterator;
class Paginator extends DoctrinePaginator
{
public const ITEMS_PER_PAGE = 5;
private int $total;
private array $data;
private int $count;
private int $totalpages;
private int $page;
public function __construct(QueryBuilder|Query $query, int $page = 1, bool $fetchJoinCollection = true)
{
$query->setFirstResult(($page - 1) * self::ITEMS_PER_PAGE);
$query->setMaxResults(self::ITEMS_PER_PAGE);
parent::__construct($query, $fetchJoinCollection);
$this->total = $this->count();
$this->data = iterator_to_array(parent::getIterator());
$this->count = count($this->data);
$this->page = $page;
try {
$this->totalpages = ceil($this->total / self::ITEMS_PER_PAGE);
} catch (\DivisionByZeroError $e) {
$this->totalpages = 0;
}
}
public function getTotal(): int
{
return $this->total;
}
public function getData(): array
{
return $this->data;
}
public function getCount(): int
{
return $this->count;
}
public function getTotalPages(): int
{
return $this->totalpages;
}
public function getCurrentPage(): int
{
return $this->page;
}
public function getItemsPerPage(): ?int
{
return $this->getQuery()->getMaxResults();
}
public function getOffset(): ?int
{
return $this->getQuery()->getFirstResult();
}
public function hasNextPage(): bool
{
if ($this->getCurrentPage() >= 1 && $this->getCurrentPage() < $this->getTotalPages()) {
return true;
}
return false;
}
public function hasPreviousPage(): bool
{
if ($this->getCurrentPage() > 1 && $this->getCurrentPage() <= $this->getTotalPages()) {
return true;
}
return false;
}
public function getIterator(): ArrayIterator
{
return new ArrayIterator([
'data' => $this->getData(),
'pagination' => [
'total' => $this->getTotal(),
'count' => $this->getCount(),
'offset' => $this->getOffset(),
'items_per_page' => $this->getItemsPerPage(),
'total_pages' => $this->getTotalPages(),
'current_page' => $this->getCurrentPage(),
'has_next_page' => $this->hasNextPage(),
'has_previous_page' => $this->hasPreviousPage(),
],
]);
}
}
As we have mentioned, our Paginator
object can be made with only 2 parameters:
- The request
- The desired page
Here, the desired page will be used to fill in the offset
. This offset
will be used in the SQL query to retrieve only the elements corresponding to the desired page.
Let's take the time to detail the different methods.
- The
getTotal
,getData
,getCount
,getTotalPages
andgetCurrentPage
methods are simplygetter
for thetotal
,data
,count
,totalPages
andpage
attributes. - The
getItemsPerPage
method is a utility method that returns the number of items per page. The value corresponds to the limit given in the SQL query. It is also the same value that is filled in theITEMS_PER_PAGE
constant. - The
getOffset
method, like the previous one, is a utility method to retrieve the offset that is filled in the SQL query. - The
hasNextPage
method allows to know if another page exists after the current page. - Conversely, the method
hasPreviousPage
allows to know if a page existed before the current page. - Finally, the most important part, the
getIterator
method allows to format the result of the pagination in the desired format. This result will be used when serializing the pagination object.
Going back to this getIterator
method, we can notice that we define two main fields: data
and pagination
.
The data
field simply lists the elements returned by the SQL query after pagination.
The pagination
field, on the other hand, allows to return information about the pagination. It is here that we find the different methods we defined previously.
Now that we have our object allowing us to paginate a query, we just have to use it in our queries.
Implementation in existing code
The use of our Paginator
is rather. We will mostly use it in repositories.
Remember that the pagination system will only be used in the case where a list of elements is returned. The pagination will not be used for modification, deletion or creation requests.
Let's take as support the /todos
request with the GET
method.
Currently, we have the following code:
// src/Controller/TodoController.php
#[Route('/todos', name: 'get_todos', methods: ["GET"])]
public function getTodos(TodoRepository $todoRepository): JsonResponse
{
$todos = $todoRepository->findAll();
return $this->json($todos);
}
Unfortunately, the result returned by the findAll
method is a list of all elements without any restriction. We'll try to use our pagination system instead. To do this, we'll have to start by creating a new method in the Todo
repository. This method will return a paged list to the controller. Let's call this new method findAllWithPagination
. Since our Paginator
object needs the desired page, we'll need to enter it into our repository.
// src/Repository/TodoRepository.php
use App\Model\Paginator;
public function findAllWithPagination(int $page): Paginator
{
// ...
}
Then, we will simply have to create our query without filling in the limit or offset because they will be handled by the Paginator
directly. In our case, we want to retrieve all the items sorted from oldest to newest.
// src/Repository/TodoRepository.php
$query = $this->createQueryBuilder('t')->orderBy('t.createdAt', 'DESC');
Then, we will just have to fill this request to our Paginator
object with the desired page, and it will be done.
// src/Repository/TodoRepository.php
return new Paginator($query, $page);
You should end up with the following method.
// src/Repository/TodoRepository.php
public function findAllWithPagination(int $page): Paginator
{
$query = $this->createQueryBuilder('t')->orderBy('t.createdAt', 'ASC');
return new Paginator($query, $page);
}
Finally, in the controller, rather than calling the findAll
method, we will instead call our new findAllWithPagination
method.
// src/Controller/TodoController.php
#[Route('/todos', name: 'get_todos', methods: ["GET"])]
public function getTodos(TodoRepository $todoRepository): JsonResponse
{
$todos = $todoRepository->findAllWithPagination(1);
return $this->json($todos);
}
Now, if we go to /api/todos
with the GET
method you should get the following result.
{
"data": [
{
"id": 1,
"title": "Updated title",
"createdAt": "2023-04-23T12:38:53+00:00",
"updatedAt": "2023-04-23T13:29:52+00:00",
"completed": false
},
{
"id": 2,
"title": "Updated title",
"createdAt": "2023-04-23T12:38:53+00:00",
"updatedAt": "2023-04-23T13:28:45+00:00",
"completed": false
},
{
"id": 3,
"title": "Et et et vero vel aut assumenda et. Voluptatem repudiandae accusantium dolor ad quae exercitationem voluptas. Voluptatem quis aspernatur sed ab laudantium sequi. Id omnis accusantium laudantium culpa vero in sunt.",
"createdAt": "2023-04-23T12:38:53+00:00",
"updatedAt": "2023-04-23T12:38:53+00:00",
"completed": false
},
{
"id": 4,
"title": "Animi velit et aut consequuntur. Tempora impedit quidem nobis explicabo nobis doloribus qui. Cumque cumque commodi illum voluptatem necessitatibus quia sed doloremque.",
"createdAt": "2023-04-23T12:38:53+00:00",
"updatedAt": "2023-04-23T12:38:53+00:00",
"completed": false
},
{
"id": 5,
"title": "Reiciendis delectus ut sed delectus. Nihil esse voluptatem qui inventore. Eos ipsam eveniet ut molestiae. Tempore nihil consequatur ut labore aliquid aliquam ut.",
"createdAt": "2023-04-23T12:38:53+00:00",
"updatedAt": "2023-04-23T12:38:53+00:00",
"completed": false
}
],
"pagination": {
"total": 6,
"count": 5,
"offset": 0,
"items_per_page": 5,
"total_pages": 2,
"current_page": 1,
"has_next_page": true,
"has_previous_page": false
}
}
Unfortunately, for the moment, this route only returns the todos of the first page. So we will have to manage the pages via the query query parameters and the page change.
Page management
In many cases, the management of pages and other parameters such as sorting or the number of elements desired, are managed via the query parameters
. These are the parameters found in the url after a ?
. For example the url /api/todos?page=2
contains the query parameter
page
which has the value 2
.
As for the validation of the data sent by the user, we will have to validate the page parameter in order to avoid errors. To do this, we will reuse the Options Resolver
component.
First, let's create our Options Resolver
. In the file /src/OptionsResolver/PaginatorOptionsResolver.php
, let us fill in the following content.
// src/OptionsResolver/PaginatorOptionsResolver.php
namespace App\OptionsResolver;
use Symfony\Component\OptionsResolver\Options;
use Symfony\Component\OptionsResolver\OptionsResolver;
class PaginatorOptionsResolver extends OptionsResolver
{
public function configurePage(): self
{
return $this
->setDefined("page")
->setDefault("page", 1)
->setAllowedTypes("page", "numeric")
->setAllowedValues("page", function ($page) {
$validatedValue = filter_var($page, FILTER_VALIDATE_INT, [
'flags' => FILTER_NULL_ON_FAILURE,
]);
if(null === $validatedValue || $validatedValue < 1) {
return false;
}
return true;
})
->setNormalizer("page", fn (Options $options, $page) => (int) $page);
}
}
The principle is similar to the previous options resolver
we were able to create. In the configurePage
method, we specify the following:
- We indicate that the
page
field exists - We set the default value of the
page
field - We specify the allowed type. Although the page is an
integer
, we cannot use theint
type because the variables from thequery parameters
are necessarily of typestring
. On the other hand, thenumeric
type allows us to determine if a string corresponds to a number. - We define the allowed values. Here, we want the numeric value to be a positive integer
- Finally we create a
normalizer
in order to format ourpage
field as an integer. Without this operation, the value of thepage
field will be of typestring
and not of typeinteger
.
To use it, as before, we just need to inject our Options Resolver
into the desired method via the method parameters and then use it.
As a reminder, we will only use paging for the /api/todos
request with the GET
HTTP method.
So we end up with the following code.
// src/Controller/TodoController.php
use App\OptionsResolver\PaginatorOptionsResolver;
#[Route('/todos', name: 'get_todos', methods: ["GET"])]
public function getTodos(TodoRepository $todoRepository, Request $request, PaginatorOptionsResolver $paginatorOptionsResolver): JsonResponse
{
try {
$queryParams = $paginatorOptionsResolver
->configurePage()
->resolve($request->query->all());
$todos = $todoRepository->findAllWithPagination($queryParams["page"]);
return $this->json($todos);
} catch(Exception $e) {
throw new BadRequestHttpException($e->getMessage());
}
}
The principle is the same as before. We validate the query parameters
from the $request->query->all()
method. Then we use page in our repository.
Finally, we use a try catch
so as to return a 400 Bad Request
error instead of 500 Internal Server Error
.
Now, if we try our /api/todos
route again with the GET
method, we end up with the same result as before. However, this time we can change the desired page by changing the page
parameter in the url.
In summary
In this lesson we learned how to create a functional pagination from the Doctrine Paginator
. We also reused the Options Resolver
component in order to validate the page parameter that the user sends us. Finally, we discovered the :
setDefault
setAllowedValues
setNormalizer
as part of the Options Resolvers
.
The functional part of our api is now complete. In other words, the returned results will not change from now on. However, the course is not finished. Indeed, we still have to implement a security system to our api via the bearer
token. This is the subject of the next lesson.