You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This library provides a Queue Worker total solution for Codeigniter 3 framework, which includes Listener and Worker for processing new jobs from queue. You may integrate your application queue (such as Redis) with Queue Worker Controller.
76
+
77
+
Listener could continue to run for detecting new jobs until it is manually stopped or you close your terminal. On the other hand
78
+
, Worker could continue to run for processing new jobs until there is no job left.
You need to design porcesses or set properties for your own worker inherited from this library, there are common interfaces as following:
111
+
You need to design handlers for your own worker inherited from this library, there are common interfaces as following:
95
112
96
113
```php
97
114
use yidas\queue\worker\Controller as WorkerController;
@@ -101,14 +118,16 @@ class My_worker extends WorkerController
101
118
// Initializer
102
119
protected function init() {}
103
120
104
-
// Listener
105
-
protected function listenerCallback() {}
106
-
107
121
// Worker
108
-
protected function workerCallback() {}
122
+
protected function handleWork() {}
123
+
124
+
// Listener
125
+
protected function handleListen() {}
109
126
}
110
127
```
111
128
129
+
These handlers are supposed to be designed for detecting the same job queue, but for different purpose. For example, Listener and Worker detect the same Redis list queue, Listener only do dispatching jobs by forking Worker, while Worker continue to takes out jobs and do the processing until job queue is empty.
130
+
112
131
### How to Design a Worker
113
132
114
133
#### 1. Build Initializer
@@ -117,57 +136,73 @@ class My_worker extends WorkerController
117
136
protected void init()
118
137
```
119
138
139
+
The `init()` method is the constructor of worker controller, it provides you with an interface for defining initializartion such as Codeigniter library loading.
140
+
120
141
*Example Code:*
121
142
```php
122
143
class My_worker extends \yidas\queue\worker\Controller
The `handleWork()` method is a processor for Worker that continue to take out jobs and do the processing. When this method returns `false`, that means the job queue is empty and the worker will close itself.
163
+
141
164
*Example Code:*
142
165
```php
143
166
class My_worker extends \yidas\queue\worker\Controller
144
167
{
145
-
protected function listenerCallback()
168
+
protected function handleWork()
146
169
{
147
-
// `true` for task existing
148
-
return $this->myqueue->exists();
170
+
$job = $this->myjobs->popJob();
171
+
172
+
// `false` for job not found, which would close the worker itself.
173
+
if (!$job)
174
+
return false;
175
+
176
+
$this->myjobs->processJob($job);
177
+
178
+
// `true` for job existing, which would keep handling.
The `handleListen()` method is a processor for Listener that dispatches workers to handle jobs while it detects new job by returning `true`. When this method returns `false`, that means the job queue is empty and the listener will stop dispatching.
191
+
159
192
*Example Code:*
160
193
```php
161
194
class My_worker extends \yidas\queue\worker\Controller
162
195
{
163
-
protected function workerCallback()
196
+
protected function handleListen()
164
197
{
165
-
// `false` for task not found
166
-
return $this->myqueue->processTask();
198
+
// `true` for job existing, which leads to dispatch worker(s).
199
+
// `false` for job not found, which would keep detecting new job
200
+
return $this->myjobs->exists();
167
201
}
168
202
// ...
169
203
```
170
204
205
+
171
206
### Porperties Setting
172
207
173
208
You could customize your worker by defining properties.
@@ -177,11 +212,10 @@ use yidas\queue\worker\Controller as WorkerController;
177
212
178
213
class My_worker extends WorkerController
179
214
{
180
-
// Set for that a listener only create a worker
181
-
// set to 1 could prevent race condition depended on your queue structure
182
-
public $workerMaxNum = 1;
215
+
// Setting for that a listener could fork up to 10 workers
216
+
public $workerMaxNum = 10;
183
217
184
-
// Enable text log writen into specified file
218
+
// Enable text log writen into specified file for listener and worker
185
219
public $logPath = 'tmp/my-worker.log';
186
220
}
187
221
```
@@ -194,28 +228,112 @@ class My_worker extends WorkerController
194
228
|$logPath |string |null |Log file path|
195
229
|$phpCommand |string |'php' |PHP CLI command for current environment|
196
230
|$listenerSleep |integer |3 |Time interval of listen frequency on idle|
197
-
|$workerSleep |integer |0 |Time interval of worker processes|
231
+
|$workerSleep |integer |0 |Time interval of worker processes frequency|
198
232
|$workerMaxNum |integer |5 |Number of max workers|
199
233
|$workerStartNum |integer |1 |Number of workers at start, less than or equal to $workerMaxNum|
200
234
|$workerWaitSeconds|integer |10 |Waiting time between worker started and next worker starting|
201
-
235
+
|$workerHeathCheck |boolean |true |Enable worker health check for listener|
202
236
203
237
---
204
238
205
239
USAGE
206
240
-----
207
241
208
-
After configurating a worker, this worker controller is ready to go:
242
+
There are 3 actions for usage:
243
+
244
+
-`listen` A listener to manage and dispatch jobs by forking workers.
245
+
-`work` A worker to process and solve jobs from queue.
246
+
-`launch` A launcher to run `listen` or `work` process in background and keep it running uniquely.
247
+
248
+
You could run above actions by using Codeigniter 3 PHP-CLI command after configuring a Queue Worker controller.
249
+
250
+
### Running Queue Worker
251
+
252
+
#### Worker
253
+
254
+
To process new jobs from the queue, you could simply run Worker:
255
+
256
+
```
257
+
$ php index.php myjob/work
258
+
```
259
+
260
+
As your worker processor `handleWork()`, the worker will continue to run (return `true`) until the job queue is empty (return `false`).
261
+
262
+
263
+
#### Listener
264
+
265
+
To start a listener to manage workers, you could simply run Listener:
209
266
210
267
```
211
-
$ php ./index.php my_worker/listener
268
+
$ php index.php myjob/listen
212
269
```
213
270
214
-
Listener would continuously process listener callback funciton, it would assign works by forking workers while the callback return `true` which means that there has task(s) detected.
271
+
As your listener processor `handleListen()`, the listener will dispatch workers when detecting new jobs (return `true`) until the job queue is empty with stopping dispatching and listening for next new jobs (return `false`).
272
+
273
+
Listener manage Workers by forking each Worker into running process, it implements Multi-Processes which could dramatically improve job queue performance.
274
+
275
+
### Running in Background
276
+
277
+
This library supports running Listener or Worker permanently in the background, it provides you the ability to run Worker as service.
278
+
279
+
#### Launcher
280
+
281
+
To run Listener or Worker in the background, you could call Launcher to launch process:
282
+
283
+
```
284
+
$ php index.php myjob/launch
285
+
```
286
+
287
+
By default, Launcher would launch `listen` process, you could also launch `work` by giving parameter:
288
+
289
+
```
290
+
$ php index.php myjob/launch/worker
291
+
```
292
+
293
+
Launcher could keep launching process running uniquely, which prevents multiple same listeners or workers running at the same time. For example, the first time to launch a listener:
294
+
295
+
```ps
296
+
$ php index.php myjob/launch
297
+
Success to launch process `listen`: myjob/listen.
298
+
Called command: php /srv/ci-project/index.php myjob/listen > /dev/null &
299
+
------
300
+
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
Then, when you launch the listener again, Launcher would prevent repeated running:
305
+
306
+
```ps
307
+
$ php index.php myjob/launch
308
+
Skip: Same process `listen` is running: myjob/listen.
309
+
------
310
+
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
311
+
user 14650 0.4 0.9 337764 36616 pts/3 S 15:43 0:00 php /srv/ci-project/index.php myjob/listen
312
+
```
313
+
314
+
For uniquely work scenario, you may use database as application queue, which would lead to race condition if there are multiple workers handling the same jobs. Unlike memcache list, database queue should be processed by only one worker at the same time.
315
+
316
+
#### Process Status
317
+
318
+
After launching a listener, you could check the listener service by command `ps aux|grep php`:
319
+
320
+
```ps
321
+
...
322
+
www-data 2278 0.7 1.0 496852 84144 ? S Sep25 37:29 php-fpm: pool www
0 commit comments