Skip to content

Commit 9c512a9

Browse files
wso2-jenkins-botwso2-jenkins-bot
committed
[WSO2-Release] [Release 2.0.2] update documentation for release 2.0.2
1 parent f08d832 commit 9c512a9

File tree

5 files changed

+281
-39
lines changed

5 files changed

+281
-39
lines changed

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,12 +19,12 @@ For information on <a target="_blank" href="https://siddhi.io/">Siddhi</a> and i
1919

2020
## Latest API Docs
2121

22-
Latest API Docs is <a target="_blank" href="https://siddhi-io.github.io/siddhi-io-file/api/2.0.1">2.0.1</a>.
22+
Latest API Docs is <a target="_blank" href="https://siddhi-io.github.io/siddhi-io-file/api/2.0.2">2.0.2</a>.
2323

2424
## Features
2525

26-
* <a target="_blank" href="https://siddhi-io.github.io/siddhi-io-file/api/2.0.1/#file-sink">file</a> *<a target="_blank" href="http://siddhi.io/en/v5.1/docs/query-guide/#sink">(Sink)</a>*<br><div style="padding-left: 1em;"><p>File Sink can be used to publish (write) event data which is processed within Siddhi to files. <br>Siddhi-io-file sink provides support to write both textual and binary data into files<br></p></div>
27-
* <a target="_blank" href="https://siddhi-io.github.io/siddhi-io-file/api/2.0.1/#file-source">file</a> *<a target="_blank" href="http://siddhi.io/en/v5.1/docs/query-guide/#source">(Source)</a>*<br><div style="padding-left: 1em;"><p>File Source provides the functionality for user to feed data to Siddhi from files. Both text and binary files are supported by file source.</p></div>
26+
* <a target="_blank" href="https://siddhi-io.github.io/siddhi-io-file/api/2.0.2/#file-sink">file</a> *(<a target="_blank" href="http://siddhi.io/en/v5.1/docs/query-guide/#sink">Sink</a>)*<br> <div style="padding-left: 1em;"><p><p style="word-wrap: break-word;margin: 0;">File Sink can be used to publish (write) event data which is processed within siddhi to files. <br>Siddhi-io-file sink provides support to write both textual and binary data into files<br></p></p></div>
27+
* <a target="_blank" href="https://siddhi-io.github.io/siddhi-io-file/api/2.0.2/#file-source">file</a> *(<a target="_blank" href="http://siddhi.io/en/v5.1/docs/query-guide/#source">Source</a>)*<br> <div style="padding-left: 1em;"><p><p style="word-wrap: break-word;margin: 0;">File Source provides the functionality for user to feed data to siddhi from files. Both text and binary files are supported by file source.</p></p></div>
2828

2929
## Dependencies
3030

‎docs/api/2.0.2.md‎

Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
# API Docs - v2.0.2
2+
3+
!!! Info "Tested Siddhi Core version: *<a target="_blank" href="http://siddhi.io/en/v5.1/docs/query-guide/">5.1.5</a>*"
4+
It could also support other Siddhi Core minor versions.
5+
6+
## Sink
7+
8+
### file *<a target="_blank" href="http://siddhi.io/en/v5.1/docs/query-guide/#sink">(Sink)</a>*
9+
<p></p>
10+
<p style="word-wrap: break-word;margin: 0;">File Sink can be used to publish (write) event data which is processed within siddhi to files. <br>Siddhi-io-file sink provides support to write both textual and binary data into files<br></p>
11+
<p></p>
12+
<span id="syntax" class="md-typeset" style="display: block; font-weight: bold;">Syntax</span>
13+
14+
```
15+
@sink(type="file", file.uri="<STRING>", append="<BOOL>", add.line.separator="<BOOL>", @map(...)))
16+
```
17+
18+
<span id="query-parameters" class="md-typeset" style="display: block; color: rgba(0, 0, 0, 0.54); font-size: 12.8px; font-weight: bold;">QUERY PARAMETERS</span>
19+
<table>
20+
<tr>
21+
<th>Name</th>
22+
<th style="min-width: 20em">Description</th>
23+
<th>Default Value</th>
24+
<th>Possible Data Types</th>
25+
<th>Optional</th>
26+
<th>Dynamic</th>
27+
</tr>
28+
<tr>
29+
<td style="vertical-align: top">file.uri</td>
30+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">Used to specify the file for data to be written. </p></td>
31+
<td style="vertical-align: top"></td>
32+
<td style="vertical-align: top">STRING</td>
33+
<td style="vertical-align: top">No</td>
34+
<td style="vertical-align: top">Yes</td>
35+
</tr>
36+
<tr>
37+
<td style="vertical-align: top">append</td>
38+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This parameter is used to specify whether the data should be append to the file or not.<br>If append = 'true', data will be write at the end of the file without changing the existing content.<br>If file does not exist, a new fill will be crated and then data will be written.<br>If append append = 'false', <br>If given file exists, existing content will be deleted and then data will be written back to the file.<br>If given file does not exist, a new file will be created and then data will be written on it.<br></p></td>
39+
<td style="vertical-align: top">true</td>
40+
<td style="vertical-align: top">BOOL</td>
41+
<td style="vertical-align: top">Yes</td>
42+
<td style="vertical-align: top">No</td>
43+
</tr>
44+
<tr>
45+
<td style="vertical-align: top">add.line.separator</td>
46+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This parameter is used to specify whether events added to the file should be separated by a newline.<br>If add.event.separator= 'true',then a newline will be added after data is added to the file.</p></td>
47+
<td style="vertical-align: top">true. (However, if csv mapper is used, it is false)</td>
48+
<td style="vertical-align: top">BOOL</td>
49+
<td style="vertical-align: top">Yes</td>
50+
<td style="vertical-align: top">No</td>
51+
</tr>
52+
</table>
53+
54+
<span id="examples" class="md-typeset" style="display: block; font-weight: bold;">Examples</span>
55+
<span id="example-1" class="md-typeset" style="display: block; color: rgba(0, 0, 0, 0.54); font-size: 12.8px; font-weight: bold;">EXAMPLE 1</span>
56+
```
57+
@sink(type='file', @map(type='json'), append='false', file.uri='/abc/{{symbol}}.txt') define stream BarStream (symbol string, price float, volume long);
58+
```
59+
<p></p>
60+
<p style="word-wrap: break-word;margin: 0;">Under above configuration, for each event, a file will be generated if there's no such a file,and then data will be written to that file as json messagesoutput will looks like below.<br>{<br>&nbsp;&nbsp;&nbsp;&nbsp;"event":{<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"symbol":"WSO2",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"price":55.6,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"volume":100<br>&nbsp;&nbsp;&nbsp;&nbsp;}<br>}<br></p>
61+
<p></p>
62+
## Source
63+
64+
### file *<a target="_blank" href="http://siddhi.io/en/v5.1/docs/query-guide/#source">(Source)</a>*
65+
<p></p>
66+
<p style="word-wrap: break-word;margin: 0;">File Source provides the functionality for user to feed data to siddhi from files. Both text and binary files are supported by file source.</p>
67+
<p></p>
68+
<span id="syntax" class="md-typeset" style="display: block; font-weight: bold;">Syntax</span>
69+
70+
```
71+
@source(type="file", dir.uri="<STRING>", file.uri="<STRING>", mode="<STRING>", tailing="<BOOL>", action.after.process="<STRING>", action.after.failure="<STRING>", move.after.process="<STRING>", move.after.failure="<STRING>", begin.regex="<STRING>", end.regex="<STRING>", file.polling.interval="<STRING>", dir.polling.interval="<STRING>", timeout="<STRING>", file.read.wait.timeout="<STRING>", @map(...)))
72+
```
73+
74+
<span id="query-parameters" class="md-typeset" style="display: block; color: rgba(0, 0, 0, 0.54); font-size: 12.8px; font-weight: bold;">QUERY PARAMETERS</span>
75+
<table>
76+
<tr>
77+
<th>Name</th>
78+
<th style="min-width: 20em">Description</th>
79+
<th>Default Value</th>
80+
<th>Possible Data Types</th>
81+
<th>Optional</th>
82+
<th>Dynamic</th>
83+
</tr>
84+
<tr>
85+
<td style="vertical-align: top">dir.uri</td>
86+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">Used to specify a directory to be processed. <br>All the files inside this directory will be processed. <br>Only one of 'dir.uri' and 'file.uri' should be provided.<br>This uri MUST have the respective protocol specified.</p></td>
87+
<td style="vertical-align: top"></td>
88+
<td style="vertical-align: top">STRING</td>
89+
<td style="vertical-align: top">No</td>
90+
<td style="vertical-align: top">No</td>
91+
</tr>
92+
<tr>
93+
<td style="vertical-align: top">file.uri</td>
94+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">Used to specify a file to be processed. <br>&nbsp;Only one of 'dir.uri' and 'file.uri' should be provided.<br>This uri MUST have the respective protocol specified.<br></p></td>
95+
<td style="vertical-align: top"></td>
96+
<td style="vertical-align: top">STRING</td>
97+
<td style="vertical-align: top">No</td>
98+
<td style="vertical-align: top">No</td>
99+
</tr>
100+
<tr>
101+
<td style="vertical-align: top">mode</td>
102+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This parameter is used to specify how files in given directory should.Possible values for this parameter are,<br>1. TEXT.FULL : to read a text file completely at once.<br>2. BINARY.FULL : to read a binary file completely at once.<br>3. LINE : to read a text file line by line.<br>4. REGEX : to read a text file and extract data using a regex.<br></p></td>
103+
<td style="vertical-align: top">line</td>
104+
<td style="vertical-align: top">STRING</td>
105+
<td style="vertical-align: top">Yes</td>
106+
<td style="vertical-align: top">No</td>
107+
</tr>
108+
<tr>
109+
<td style="vertical-align: top">tailing</td>
110+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This can either have value true or false. By default it will be true. <br>This attribute allows user to specify whether the file should be tailed or not. <br>If tailing is enabled, the first file of the directory will be tailed.<br>Also tailing should not be enabled in 'binary.full' or 'text.full' modes.<br></p></td>
111+
<td style="vertical-align: top">true</td>
112+
<td style="vertical-align: top">BOOL</td>
113+
<td style="vertical-align: top">Yes</td>
114+
<td style="vertical-align: top">No</td>
115+
</tr>
116+
<tr>
117+
<td style="vertical-align: top">action.after.process</td>
118+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This parameter is used to specify the action which should be carried out <br>after processing a file in the given directory. <br>It can be either DELETE or MOVE and default value will be 'DELETE'.<br>If the action.after.process is MOVE, user must specify the location to move consumed files using 'move.after.process' parameter.<br></p></td>
119+
<td style="vertical-align: top">delete</td>
120+
<td style="vertical-align: top">STRING</td>
121+
<td style="vertical-align: top">Yes</td>
122+
<td style="vertical-align: top">No</td>
123+
</tr>
124+
<tr>
125+
<td style="vertical-align: top">action.after.failure</td>
126+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This parameter is used to specify the action which should be carried out if a failure occurred during the process. <br>It can be either DELETE or MOVE and default value will be 'DELETE'.<br>If the action.after.failure is MOVE, user must specify the location to move consumed files using 'move.after.failure' parameter.<br></p></td>
127+
<td style="vertical-align: top">delete</td>
128+
<td style="vertical-align: top">STRING</td>
129+
<td style="vertical-align: top">Yes</td>
130+
<td style="vertical-align: top">No</td>
131+
</tr>
132+
<tr>
133+
<td style="vertical-align: top">move.after.process</td>
134+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">If action.after.process is MOVE, user must specify the location to move consumed files using 'move.after.process' parameter.<br>This should be the absolute path of the file that going to be created after moving is done.<br>This uri MUST have the respective protocol specified.<br></p></td>
135+
<td style="vertical-align: top"></td>
136+
<td style="vertical-align: top">STRING</td>
137+
<td style="vertical-align: top">No</td>
138+
<td style="vertical-align: top">No</td>
139+
</tr>
140+
<tr>
141+
<td style="vertical-align: top">move.after.failure</td>
142+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">If action.after.failure is MOVE, user must specify the location to move consumed files using 'move.after.failure' parameter.<br>This should be the absolute path of the file that going to be created after moving is done.<br>This uri MUST have the respective protocol specified.<br></p></td>
143+
<td style="vertical-align: top"></td>
144+
<td style="vertical-align: top">STRING</td>
145+
<td style="vertical-align: top">No</td>
146+
<td style="vertical-align: top">No</td>
147+
</tr>
148+
<tr>
149+
<td style="vertical-align: top">begin.regex</td>
150+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This will define the regex to be matched at the beginning of the retrieved content.<br></p></td>
151+
<td style="vertical-align: top">None</td>
152+
<td style="vertical-align: top">STRING</td>
153+
<td style="vertical-align: top">Yes</td>
154+
<td style="vertical-align: top">No</td>
155+
</tr>
156+
<tr>
157+
<td style="vertical-align: top">end.regex</td>
158+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This will define the regex to be matched at the end of the retrieved content.<br></p></td>
159+
<td style="vertical-align: top">None</td>
160+
<td style="vertical-align: top">STRING</td>
161+
<td style="vertical-align: top">Yes</td>
162+
<td style="vertical-align: top">No</td>
163+
</tr>
164+
<tr>
165+
<td style="vertical-align: top">file.polling.interval</td>
166+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This parameter is used to specify the time period (in milliseconds) of a polling cycle for a file.<br></p></td>
167+
<td style="vertical-align: top">1000</td>
168+
<td style="vertical-align: top">STRING</td>
169+
<td style="vertical-align: top">Yes</td>
170+
<td style="vertical-align: top">No</td>
171+
</tr>
172+
<tr>
173+
<td style="vertical-align: top">dir.polling.interval</td>
174+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This parameter is used to specify the time period (in milliseconds) of a polling cycle for a directory.<br></p></td>
175+
<td style="vertical-align: top">1000</td>
176+
<td style="vertical-align: top">STRING</td>
177+
<td style="vertical-align: top">Yes</td>
178+
<td style="vertical-align: top">No</td>
179+
</tr>
180+
<tr>
181+
<td style="vertical-align: top">timeout</td>
182+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This parameter is used to specify the maximum time period (in milliseconds) for waiting until a file is processed.<br></p></td>
183+
<td style="vertical-align: top">5000</td>
184+
<td style="vertical-align: top">STRING</td>
185+
<td style="vertical-align: top">Yes</td>
186+
<td style="vertical-align: top">No</td>
187+
</tr>
188+
<tr>
189+
<td style="vertical-align: top">file.read.wait.timeout</td>
190+
<td style="vertical-align: top; word-wrap: break-word"><p style="word-wrap: break-word;margin: 0;">This parameter is used to specify the maximum time period (in milliseconds) till it waits before retrying to read the full file content.</p></td>
191+
<td style="vertical-align: top">1000</td>
192+
<td style="vertical-align: top">STRING</td>
193+
<td style="vertical-align: top">Yes</td>
194+
<td style="vertical-align: top">No</td>
195+
</tr>
196+
</table>
197+
198+
<span id="examples" class="md-typeset" style="display: block; font-weight: bold;">Examples</span>
199+
<span id="example-1" class="md-typeset" style="display: block; color: rgba(0, 0, 0, 0.54); font-size: 12.8px; font-weight: bold;">EXAMPLE 1</span>
200+
```
201+
@source(type='file',
202+
mode='text.full',
203+
tailing='false'
204+
dir.uri='file://abc/xyz',
205+
action.after.process='delete',
206+
@map(type='json'))
207+
define stream FooStream (symbol string, price float, volume long);
208+
209+
```
210+
<p></p>
211+
<p style="word-wrap: break-word;margin: 0;">Under above configuration, all the files in directory will be picked and read one by one.<br>In this case, it's assumed that all the files contains json valid json strings with keys 'symbol','price' and 'volume'.<br>Once a file is read, its content will be converted to an event using siddhi-map-json extension and then, that event will be received to the FooStream.<br>Finally, after reading is finished, the file will be deleted.<br></p>
212+
<p></p>
213+
<span id="example-2" class="md-typeset" style="display: block; color: rgba(0, 0, 0, 0.54); font-size: 12.8px; font-weight: bold;">EXAMPLE 2</span>
214+
```
215+
@source(type='file',
216+
mode='files.repo.line',
217+
tailing='true',
218+
dir.uri='file://abc/xyz',
219+
@map(type='json'))
220+
define stream FooStream (symbol string, price float, volume long);
221+
222+
```
223+
<p></p>
224+
<p style="word-wrap: break-word;margin: 0;">Under above configuration, the first file in directory '/abc/xyz' will be picked and read line by line.<br>In this case, it is assumed that the file contains lines json strings.<br>For each line, line content will be converted to an event using siddhi-map-json extension and then, that event will be received to the FooStream.<br>Once file content is completely read, it will keep checking whether a new entry is added to the file or not.<br>If such entry is added, it will be immediately picked up and processed.<br></p>
225+
<p></p>

0 commit comments

Comments
 (0)