Skip to content

Commit 63081e3

Browse files
guidiazguidiaz
committed
chore: first written draft on http-head proposal
1 parent ac05b17 commit 63081e3

1 file changed

Lines changed: 161 additions & 0 deletions

File tree

‎draft-guidiaz-http-head.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
<pre>
2+
WIP: WIP-0029
3+
Layer: Consensus (hard fork)
4+
Title: Support HTTP-HEAD in RADON
5+
Authors: Guillermo Díaz <guillermo@otherplane.com>
6+
Discussions-To: ..tbd..
7+
Status: Draft
8+
Type: Standards Track
9+
Created: 2024-02-22
10+
License: BSD-2-Clause
11+
</pre>
12+
13+
14+
## Abstract
15+
16+
Data Requests in the Witnet blockchain currently support the aggregation of data extracted from any combination of HTTP-GET and HTTP-POST retrievals. This proposal specifies the support of HTTP-HEAD data retrievals, enabling data requests to extract metadata about pre-existing content on the Internet.
17+
18+
## Motivation and rationale
19+
20+
While HTTP-GET and HTTP-POST data retrievals may cover the majority of use cases, other potential applications rather need to retrieve metadata from allegedly existing resource contents on the Internet, instead of relyng on calls to REST/API endpoints.
21+
22+
For instance, metadata like the existence, age, size, hash or the content type of some given URL, can be extracted by a single HTTP-HEAD retrieval, without actually forcing witnessing nodes to download the whole content that is being targeted.
23+
24+
## Specification
25+
26+
At the protocol level, these are the new fields in the protobuf schema:
27+
28+
```diff
29+
enum RADType {
30+
Unknown = 0;
31+
HttpGet = 1;
32+
Rng = 2;
33+
HttpPost = 3;
34+
+ HttpHead = 4;
35+
}
36+
```
37+
38+
### HTTP-HEAD
39+
40+
#### Optional `headers` field
41+
42+
Just like HTTP-GET and HTTP-POST retrievals, this new retrieval type will also accept including request headers.
43+
44+
The `headers` field may contain a list of `(key, value)` pairs, using the `StringPair` helper message:
45+
46+
```
47+
message StringPair {
48+
string left = 1;
49+
string right = 2;
50+
}
51+
```
52+
53+
These pairs will be used as additional headers for the underlying HTTP request. At the protocol level, they can be any string, but the RAD engine will validate them according to the HTTP specification, and resolve the retrieval source to a `RadonError` in case of error.
54+
55+
56+
#### Transaction weights
57+
58+
The weight of a HTTP-HEAD retrievals are calculated just the same as HTTP-GET ones:
59+
60+
```
61+
RADRetrieve weight = (script bytes) + (url bytes) + (1) + (body bytes) + (headers_weight)
62+
headers_weight = for each header: (key bytes) + (value bytes) + overhead
63+
overhead = 6
64+
```
65+
66+
The overhead refers to the serialization overhead of `StringPair`, in an attempt to make the weight match the serialized size as close as possible.
67+
68+
#### HTTP-HEAD response format
69+
70+
Witnet nodes performing HTTP-HEAD retrieval will pass as input value to the corresponding Radon script a stringified JSON object containing one entry per response header as returned from the remote host.
71+
72+
For instance, whereas the HTTP-HEAD request to https://www.wikipedia.org/static/favicon/wikipedia.ico returns these lines:
73+
```
74+
HTTP/1.1 200 OK
75+
date: Thu, 22 Feb 2024 16:23:24 GMT
76+
server: mw-web.eqiad.main-f5dfc99f8-7mnkk
77+
last-modified: Wed, 29 Nov 2023 14:11:57 GMT
78+
cache-control: max-age=31536000
79+
expires: Fri, 21 Feb 2025 16:23:24 GMT
80+
access-control-allow-origin: *
81+
content-type: image/vnd.microsoft.icon
82+
etag: W/"aae-60b4b1e2d4540"
83+
vary: Accept-Encoding
84+
age: 4894
85+
x-cache: cp6012 miss, cp6009 hit/151018
86+
x-cache-status: hit-front
87+
server-timing: cache;desc="hit-front", host;desc="cp6009"
88+
strict-transport-security: max-age=106384710; includeSubDomains; preload
89+
x-client-ip: 90.167.7.109
90+
accept-ranges: bytes
91+
content-length: 2734
92+
```
93+
94+
The `RadonString` value passed as input to the Radon script would rather be encoded as:
95+
96+
```
97+
{
98+
"date": "Thu, 22 Feb 2024 16:23:24 GMT",
99+
"server": "mw-web.eqiad.main-f5dfc99f8-7mnkk",
100+
"last-modified": "Wed, 29 Nov 2023 14:11:57 GMT",
101+
"cache-control": "max-age=31536000",
102+
"expires": "Fri, 21 Feb 2025 16:23:24 GMT",
103+
"access-control-allow-origin": "*",
104+
"content-type": "image/vnd.microsoft.icon",
105+
"etag": "W/\"aae-60b4b1e2d4540\"",
106+
"content-encoding": "gzip",
107+
"vary": "Accept-Encoding",
108+
"age": "4802",
109+
"x-cache": "cp6012 miss, cp6009 hit/148083",
110+
"x-cache-status": "hit-front",
111+
"server-timing": "cache;desc=\"hit-front\", host;desc=\"cp6009\"",
112+
"strict-transport-security": "max-age=106384710; includeSubDomains; preload",
113+
"x-client-ip": "90.167.7.109",
114+
"accept-ranges": "bytes",
115+
"content-length": "1035"
116+
}
117+
```
118+
119+
## Backwards compatibility
120+
121+
- Implementer: a client that implements or adopts the protocol improvements proposed in this document.
122+
- Non-implementer: a client that fails to or refuses to implement the protocol improvements proposed in this document.
123+
124+
125+
#### Consensus cheat sheet
126+
127+
Upon entry into force of the proposed improvements:
128+
129+
- Blocks and transactions that were considered valid by former versions of the protocol MUST continue to be considered valid.
130+
- Blocks and transactions produced by non-implementers MUST be validated by implementers using the same rules as with those coming from implementers, that is, the new rules.
131+
- Implementers MUST apply the proposed logic when evaluating transactions or computing their own data request eligibility.
132+
133+
As a result:
134+
135+
- Non-implementers MAY NOT get their blocks and transactions accepted by implementers.
136+
- Implementers MAY NOT get their valid blocks accepted by non-implementers.
137+
- Non-implementers MAY consolidate different blocks and superblocks than implementers.
138+
139+
Due to this last point, this MUST be considered a consensus-critical protocol improvement. An adoption plan MUST be proposed, setting an activation date that gives miners enough time in advance to upgrade their existing clients.
140+
141+
142+
### Libraries, clients, and interfaces
143+
144+
From the perspective of a data request creator this is a backwards compatible change: all the data requests that were valid before this change will still be valid after this change.
145+
146+
All the libraries that allow creation or visualization of data requests will need to be updated in order to support the new functionality.
147+
148+
## Reference Implementation
149+
150+
A reference implementation for the proposed protocol improvement can be found in the [http-head](...) branch of the [witnet-rust] repository.
151+
152+
153+
## Adoption Plan
154+
155+
TBD...
156+
157+
## Acknowledgements
158+
159+
This proposal has been cooperatively discussed and devised by individuals from the Witnet development community.
160+
161+
[witnet-rust]: https://github.com/witnet/witnet-rust/

0 commit comments

Comments
 (0)