1.. SPDX-License-Identifier: GPL-2.0
2
3=============
4Page Pool API
5=============
6
7The page_pool allocator is optimized for the XDP mode that uses one frame
8per-page, but it can fallback on the regular page allocator APIs.
9
10Basic use involves replacing alloc_pages() calls with the
11page_pool_alloc_pages() call.  Drivers should use page_pool_dev_alloc_pages()
12replacing dev_alloc_pages().
13
14API keeps track of inflight pages, in order to let API user know
15when it is safe to free a page_pool object.  Thus, API users
16must run page_pool_release_page() when a page is leaving the page_pool or
17call page_pool_put_page() where appropriate in order to maintain correct
18accounting.
19
20API user must call page_pool_put_page() once on a page, as it
21will either recycle the page, or in case of refcnt > 1, it will
22release the DMA mapping and inflight state accounting.
23
24Architecture overview
25=====================
26
27.. code-block:: none
28
29    +------------------+
30    |       Driver     |
31    +------------------+
32            ^
33            |
34            |
35            |
36            v
37    +--------------------------------------------+
38    |                request memory              |
39    +--------------------------------------------+
40        ^                                  ^
41        |                                  |
42        | Pool empty                       | Pool has entries
43        |                                  |
44        v                                  v
45    +-----------------------+     +------------------------+
46    | alloc (and map) pages |     |  get page from cache   |
47    +-----------------------+     +------------------------+
48                                    ^                    ^
49                                    |                    |
50                                    | cache available    | No entries, refill
51                                    |                    | from ptr-ring
52                                    |                    |
53                                    v                    v
54                          +-----------------+     +------------------+
55                          |   Fast cache    |     |  ptr-ring cache  |
56                          +-----------------+     +------------------+
57
58API interface
59=============
60The number of pools created **must** match the number of hardware queues
61unless hardware restrictions make that impossible. This would otherwise beat the
62purpose of page pool, which is allocate pages fast from cache without locking.
63This lockless guarantee naturally comes from running under a NAPI softirq.
64The protection doesn't strictly have to be NAPI, any guarantee that allocating
65a page will cause no race conditions is enough.
66
67* page_pool_create(): Create a pool.
68    * flags:      PP_FLAG_DMA_MAP, PP_FLAG_DMA_SYNC_DEV
69    * order:      2^order pages on allocation
70    * pool_size:  size of the ptr_ring
71    * nid:        preferred NUMA node for allocation
72    * dev:        struct device. Used on DMA operations
73    * dma_dir:    DMA direction
74    * max_len:    max DMA sync memory size
75    * offset:     DMA address offset
76
77* page_pool_put_page(): The outcome of this depends on the page refcnt. If the
78  driver bumps the refcnt > 1 this will unmap the page. If the page refcnt is 1
79  the allocator owns the page and will try to recycle it in one of the pool
80  caches. If PP_FLAG_DMA_SYNC_DEV is set, the page will be synced for_device
81  using dma_sync_single_range_for_device().
82
83* page_pool_put_full_page(): Similar to page_pool_put_page(), but will DMA sync
84  for the entire memory area configured in area pool->max_len.
85
86* page_pool_recycle_direct(): Similar to page_pool_put_full_page() but caller
87  must guarantee safe context (e.g NAPI), since it will recycle the page
88  directly into the pool fast cache.
89
90* page_pool_release_page(): Unmap the page (if mapped) and account for it on
91  inflight counters.
92
93* page_pool_dev_alloc_pages(): Get a page from the page allocator or page_pool
94  caches.
95
96* page_pool_get_dma_addr(): Retrieve the stored DMA address.
97
98* page_pool_get_dma_dir(): Retrieve the stored DMA direction.
99
100Coding examples
101===============
102
103Registration
104------------
105
106.. code-block:: c
107
108    /* Page pool registration */
109    struct page_pool_params pp_params = { 0 };
110    struct xdp_rxq_info xdp_rxq;
111    int err;
112
113    pp_params.order = 0;
114    /* internal DMA mapping in page_pool */
115    pp_params.flags = PP_FLAG_DMA_MAP;
116    pp_params.pool_size = DESC_NUM;
117    pp_params.nid = NUMA_NO_NODE;
118    pp_params.dev = priv->dev;
119    pp_params.dma_dir = xdp_prog ? DMA_BIDIRECTIONAL : DMA_FROM_DEVICE;
120    page_pool = page_pool_create(&pp_params);
121
122    err = xdp_rxq_info_reg(&xdp_rxq, ndev, 0);
123    if (err)
124        goto err_out;
125
126    err = xdp_rxq_info_reg_mem_model(&xdp_rxq, MEM_TYPE_PAGE_POOL, page_pool);
127    if (err)
128        goto err_out;
129
130NAPI poller
131-----------
132
133
134.. code-block:: c
135
136    /* NAPI Rx poller */
137    enum dma_data_direction dma_dir;
138
139    dma_dir = page_pool_get_dma_dir(dring->page_pool);
140    while (done < budget) {
141        if (some error)
142            page_pool_recycle_direct(page_pool, page);
143        if (packet_is_xdp) {
144            if XDP_DROP:
145                page_pool_recycle_direct(page_pool, page);
146        } else (packet_is_skb) {
147            page_pool_release_page(page_pool, page);
148            new_page = page_pool_dev_alloc_pages(page_pool);
149        }
150    }
151
152Driver unload
153-------------
154
155.. code-block:: c
156
157    /* Driver unload */
158    page_pool_put_full_page(page_pool, page, false);
159    xdp_rxq_info_unreg(&xdp_rxq);
160