Commit Graph

922 Commits

Author SHA1 Message Date
ljedrz
4877898ffd chore: improve logging and tests for subscriptions
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-15 13:52:37 +02:00
ljedrz
9edce33628 fix: properly compare Subscriptions
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-15 13:52:37 +02:00
ljedrz
0635c57b8a chore: remove some logs
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-15 12:23:01 +02:00
ljedrz
5fe6fa9b70 fix: don't send out received wanted Cids if we already have them in the wantlist
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-15 11:40:58 +02:00
ljedrz
bf031f552e fix: reduce locking in SubscriptionRegistry::shutdown
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-15 10:33:42 +02:00
ljedrz
762707e940 feat: add subscription docs
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-14 17:26:46 +02:00
Joonas Koivunen
4c58cfc12b refactor: learn about the /add progress, document 2020-07-14 17:15:28 +03:00
ljedrz
0bdbf85911 fix: make the wantlist conformace tests pass
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-14 16:15:18 +02:00
Joonas Koivunen
63056a5d82 fix: clippy, avoid a clone 2020-07-14 17:13:49 +03:00
ljedrz
19a77616d7 refactor: keep subscription accounting within the registry
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-14 16:04:28 +02:00
Joonas Koivunen
152086f146 fix: hook the /add impl up 2020-07-14 16:50:25 +03:00
Joonas Koivunen
d54b3b78fa chore: minor reformatting for multipart body 2020-07-14 16:09:19 +03:00
Joonas Koivunen
af3753e25e test: discovered new failing adder test case 2020-07-14 16:09:19 +03:00
Joonas Koivunen
cfec7584b6 feat: single file /add
doesnt really handle anything gracefully yet.
2020-07-14 16:09:02 +03:00
ljedrz
25ee479ff3 fix: always update the waker in case of Pending in the subscription poll
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-14 14:20:14 +02:00
ljedrz
415a7e9cf8 fix: make the wantlist cancellation test work on separate tasks
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-14 13:58:30 +02:00
ljedrz
3db431e852 fix: remove the sync::Mutex over the SubscriptionRegistry
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-14 11:43:11 +02:00
ljedrz
3d564526e8 fix: make wantlist conformance tests pass
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-14 11:43:11 +02:00
ljedrz
8ef92259e7 feat: improve the wantlist cancellation test
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-13 15:43:42 +02:00
ljedrz
7cbf65a373 fix: use Cid v0 for subscriptions
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-13 15:43:42 +02:00
ljedrz
0d36e9c149 fix: improve the Drop impl for SubscriptionFuture
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-13 15:43:42 +02:00
ljedrz
a5f17bb4ed perf: remove an unneeded to_vec()
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-13 15:43:42 +02:00
ljedrz
23ee954bc1 perf: don't use format! where to_string() is enough
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-13 15:43:42 +02:00
ljedrz
a9f763cf97 feat: cancel Blocks from the wantlist when all their Subscriptions die
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-13 15:43:42 +02:00
bors[bot]
c257c88faf
Merge #235
235: chore: use released mpart-async r=koivunej a=koivunej

0.4.1 with fix_7 was released just now.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-07-10 08:04:27 +00:00
Joonas Koivunen
a890964a39 chore: use released mpart-async 2020-07-10 09:31:09 +03:00
bors[bot]
4e6fbdbc58
Merge #233
233: Add http timeouts r=ljedrz a=koivunej

Related to #228, this adds timeout handling to:

 * `block/get`
 * `dag/resolve`
 * `refs` (partial)
 * `cat` (partial)
 * `get` (partial)

Partial means only the initial step of at minimum path walking has the timeout. We cannot yet return errors from streamed bodies because of the lack of `Trailer` support in hyper, so no "request timeout" is enforced once we get to streaming the body.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
Co-authored-by: Joonas Koivunen <joonas.koivunen@gmail.com>
2020-07-09 15:15:39 +00:00
Joonas Koivunen
f75af14d3f
doc: fix typos
Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-07-09 18:13:49 +03:00
Joonas Koivunen
d00eb829f9 refactor: simplify maybe_timeout 2020-07-09 18:12:48 +03:00
Joonas Koivunen
a7e10a17d8 doc: StringSerialized, add deref(mut)? 2020-07-09 17:52:47 +03:00
Joonas Koivunen
741d81a7ba refactor: inline StatResponse 2020-07-09 17:31:16 +03:00
Joonas Koivunen
2034806e98 refactor: unify "GetQuery" + "StatQuery" = GetStatOptions 2020-07-09 17:29:52 +03:00
Joonas Koivunen
c43b9c6306 feat: naive timeout (with FIXMEs) for /get and /cat
the issue is similar to with refs, we dont have anything overarching the
whole request.
2020-07-09 16:30:49 +03:00
Joonas Koivunen
a1b6e3d594 feat: timeout on initial refs get_blocks 2020-07-09 16:15:15 +03:00
Joonas Koivunen
603a15a6c4 feat: timeout on dag/resolve 2020-07-09 16:14:40 +03:00
bors[bot]
a72a5f55ce
Merge #229
229: Finishing subscriptions should update the wantlist r=koivunej a=ljedrz

This can be achieved by taking advantage of the `RepoEvent::ProvideBlock` that always follows a finished subscription.

In addition there are a few other self-contained drive-by changes, described in their respective commit messages.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-07-09 12:18:16 +00:00
Joonas Koivunen
62e0986c41 feat: timeout on block/stat
this wasn't actually needed to pass the test, unsure why.
2020-07-09 14:17:41 +03:00
Joonas Koivunen
fbe90ab8ec feat: timeout on block/get 2020-07-09 14:15:44 +03:00
Joonas Koivunen
fe584f3229 add: maybe timeouted wrapper 2020-07-09 14:15:21 +03:00
Joonas Koivunen
40c7b414e3 add: serde Display/FromStr wrapper
not sure why this isn't included in serde?
2020-07-09 14:14:23 +03:00
bors[bot]
8437373df1
Merge #232
232: refactor: delay boxing after cid generation r=ljedrz a=koivunej

boxing reallocates and cid generation can fail. also added comments.

Follow-up on https://github.com/rs-ipfs/rust-ipfs/pull/230#discussion_r452031610

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-07-09 10:02:18 +00:00
Joonas Koivunen
398cc3d0b7 refactor: delay boxing after cid generation
boxing reallocates and cid generation can fail. also added comments.

Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-07-09 11:49:24 +03:00
bors[bot]
e37c9ab8d5
Merge #231
231: Streaming multiparts: pubsub r=ljedrz a=koivunej

This was seen originally but at least it seemed some months ago that there wasn't a way to process multiparts streaming. Luckily the older js-ipfs-http-client puts the message in query. Later versions related to #228 use multipart bodies, but the topic is still a query argument.

Depends on #230.

 * Adds test for the older query part message parsing
 * Uses `&'static str` instead of `&'static [u8]` (fixes error messages)

The latest version should leave us failing only the conformance tests related to:

* pinning
* `?timeout=<not sure what unit>`
  * refs
  * cat
  * get
  * block/get
  * dag/get
* wantlist

This is:

```
  146 passing (3m)
  21 pending
  8 failing
```


Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-07-09 08:27:56 +00:00
bors[bot]
c3b92c7640
Merge #230
230: Streaming multiparts at block/put and dag/put r=ljedrz a=koivunej

Supercedes #225 to cover `dag/put`. I'll close the earlier and keep this as a draft while the temp git dependency is in place.

This allows accepting multipart bodies without Content-Length header by using mpart-async. Uploads are limited to 1MB.

As minor fixes this includes checking `dag/put?input-enc=raw` as we don't support any other input-enc at the moment (rust-ipld could read dag-json, this was implemented during phase1 already).

As more minor fixes this includes transformation of LengthRequired rejection to "Missing header: content-length" instead of "UNHANDLED REJECTION".

Fix is needed for #228.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-07-09 08:04:44 +00:00
Joonas Koivunen
7498e7ebda doc: remove unneeded allows, comments 2020-07-08 22:10:55 +03:00
Joonas Koivunen
ce939c3036 fix: wanted error for missing pubsub message 2020-07-08 21:34:13 +03:00
Joonas Koivunen
1c9b31cf76 fix: get pubsub/pub message from multipart body 2020-07-08 21:09:54 +03:00
Joonas Koivunen
6ddd7774ea fix: bad use of [u8] where str would do 2020-07-08 20:33:13 +03:00
Joonas Koivunen
7637874087 refactor: reorder pub fn before inner impl 2020-07-08 19:42:47 +03:00
Joonas Koivunen
6f2bf990cb fix: check input-enc in dag/put
this is minor change but good for interop. the conformance tests only
exercise the input-enc=raw as js-ipfs-http-client does most of the heavy
work. not checking input-enc could end up someone trying to put a json
dag document expecting it to be transformed to cbor, which we currently
do not support.
2020-07-08 19:26:01 +03:00