148 Commits

Author SHA1 Message Date
Joonas Koivunen
40c7b414e3 add: serde Display/FromStr wrapper
not sure why this isn't included in serde?
2020-07-09 14:14:23 +03:00
bors[bot]
e37c9ab8d5
Merge #231
231: Streaming multiparts: pubsub r=ljedrz a=koivunej

This was seen originally but at least it seemed some months ago that there wasn't a way to process multiparts streaming. Luckily the older js-ipfs-http-client puts the message in query. Later versions related to #228 use multipart bodies, but the topic is still a query argument.

Depends on #230.

 * Adds test for the older query part message parsing
 * Uses `&'static str` instead of `&'static [u8]` (fixes error messages)

The latest version should leave us failing only the conformance tests related to:

* pinning
* `?timeout=<not sure what unit>`
  * refs
  * cat
  * get
  * block/get
  * dag/get
* wantlist

This is:

```
  146 passing (3m)
  21 pending
  8 failing
```


Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-07-09 08:27:56 +00:00
bors[bot]
c3b92c7640
Merge #230
230: Streaming multiparts at block/put and dag/put r=ljedrz a=koivunej

Supercedes #225 to cover `dag/put`. I'll close the earlier and keep this as a draft while the temp git dependency is in place.

This allows accepting multipart bodies without Content-Length header by using mpart-async. Uploads are limited to 1MB.

As minor fixes this includes checking `dag/put?input-enc=raw` as we don't support any other input-enc at the moment (rust-ipld could read dag-json, this was implemented during phase1 already).

As more minor fixes this includes transformation of LengthRequired rejection to "Missing header: content-length" instead of "UNHANDLED REJECTION".

Fix is needed for #228.

Co-authored-by: Joonas Koivunen <joonas@equilibrium.co>
2020-07-09 08:04:44 +00:00
Joonas Koivunen
7498e7ebda doc: remove unneeded allows, comments 2020-07-08 22:10:55 +03:00
Joonas Koivunen
ce939c3036 fix: wanted error for missing pubsub message 2020-07-08 21:34:13 +03:00
Joonas Koivunen
1c9b31cf76 fix: get pubsub/pub message from multipart body 2020-07-08 21:09:54 +03:00
Joonas Koivunen
6ddd7774ea fix: bad use of [u8] where str would do 2020-07-08 20:33:13 +03:00
Joonas Koivunen
7637874087 refactor: reorder pub fn before inner impl 2020-07-08 19:42:47 +03:00
Joonas Koivunen
6f2bf990cb fix: check input-enc in dag/put
this is minor change but good for interop. the conformance tests only
exercise the input-enc=raw as js-ipfs-http-client does most of the heavy
work. not checking input-enc could end up someone trying to put a json
dag document expecting it to be transformed to cbor, which we currently
do not support.
2020-07-08 19:26:01 +03:00
Joonas Koivunen
fef741c750 fix: allow multipart bodies without content-len in dag/put 2020-07-08 19:25:43 +03:00
Joonas Koivunen
4101383855 refactor: move try_only_named.. to support 2020-07-08 19:24:17 +03:00
Joonas Koivunen
087776bda1 refactor: try_only_named_multipart out of put_block 2020-07-08 19:17:58 +03:00
Joonas Koivunen
06ba40e8f2 fix: add descriptive error for LengthRequired 2020-07-08 16:03:10 +03:00
Joonas Koivunen
0a9ee274ca temp: upgrade to mpart-async with fix 2020-07-08 13:52:01 +03:00
Joonas Koivunen
76486f8ecb fix: rewrite block/put to support streaming multiparts
streaming multiparts are something we dont yet support, which is keeping
us back in an earlier version of js-ipfs-http-client for conformance
tests.
2020-07-08 13:52:01 +03:00
ljedrz
fb1742a38e refactor: minify empty structs
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-07-08 10:45:26 +02:00
ljedrz
3eade047b4 refactor: use unwrap_or_default instead of a handwritten empty slice
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-06-29 14:42:33 +02:00
ljedrz
67641dc5e0 perf: remove an unneeded clone
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-06-29 14:26:43 +02:00
ljedrz
a57d49216a perf: faster Bytes conversions in http::{block, dag}
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-06-29 13:38:32 +02:00
ljedrz
1c299109d2 refactor: shorten a match expression
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-06-29 12:41:29 +02:00
ljedrz
8c92c34d44 perf: remove an unneeded map presence check
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-06-29 12:10:23 +02:00
bors[bot]
008693fcd1
Merge #210
210: A round of cleanups r=koivunej a=ljedrz

Assorted drive-by cleanups and small refactorings; individual commits describe the specific changes.

Co-authored-by: ljedrz <ljedrz@gmail.com>
2020-06-29 08:43:47 +00:00
Joonas Koivunen
50ae390e66 fix: return "n/a" in verbose swarm peers response
this follows the lead of js-ipfs with the "n/a".
2020-06-26 18:17:42 +03:00
ljedrz
a8103cb14c refactor: simplify a From impl for PubsubHttpApiMessage
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-06-26 15:27:07 +02:00
ljedrz
2603532027 refactor: make pubsub methods move their params
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-06-26 15:11:05 +02:00
ljedrz
8c89714da9 refactor: simplify one match expression
Signed-off-by: ljedrz <ljedrz@gmail.com>
2020-06-26 15:11:05 +02:00
Joonas Koivunen
d1cd15b4ec
Merge pull request #193 from eqlabs/fix_use_tempfile
fix test using non-unique tempfile
2020-06-18 01:16:08 +03:00
Joonas Koivunen
3ba5ddf1a3 fix test using non-unique tempfile 2020-06-18 00:55:11 +03:00
Joonas Koivunen
22aaec2377 refactor: remove unshared use
unshare was required as hyper was forced to require the body stream to
be Sync due to compiler issue while it was not really needed. Latest
hyper now has this workaround built-in.
2020-06-17 23:07:39 +03:00
Joonas Koivunen
27085b3095 chore: update hyper and warp to latest 2020-06-17 23:07:39 +03:00
Joonas Koivunen
3bfa506f69 refactor: suggestions from code review
Including mostly comment fixes and removal of an extra &mut.

Co-authored-by: ljedrz <ljedrz@users.noreply.github.com>
2020-06-17 17:27:05 +03:00
Joonas Koivunen
9aa9322a33 refactor: remove unused warnings on windows 2020-06-17 17:27:05 +03:00
Joonas Koivunen
24e4c4da42 chore: cargo fmt 2020-06-17 17:27:05 +03:00
Joonas Koivunen
0d2dd477e9 doc: refresh and check out cargo docs 2020-06-17 17:27:05 +03:00
Joonas Koivunen
c874e1165a fix: windows build 2020-06-17 17:27:05 +03:00
Joonas Koivunen
6be2556341 chore: cargo fmt 2020-06-17 17:27:05 +03:00
Joonas Koivunen
f5ada8728d add(http): multiblock file /get test
this does not hit the buffer cycling cases but hits all other file
cases.
2020-06-17 17:27:05 +03:00
Joonas Koivunen
7e0fcd05b3 refactor(http): more /get test support 2020-06-17 17:27:05 +03:00
Joonas Koivunen
bd6a974db5 refactor: simplify to simple buffer in TarHelper
two buffers would allow better reusing, but only with concurrency.
2020-06-17 17:27:05 +03:00
Joonas Koivunen
6eb276eb28 add: test /get for long and file names 2020-06-17 17:27:05 +03:00
Joonas Koivunen
9151834ba3 chore: cleanup warnings, fmt 2020-06-17 17:27:05 +03:00
Joonas Koivunen
350bb9ccf7 fix: hopefully get windows compiling 2020-06-17 17:27:05 +03:00
Joonas Koivunen
92cc2ab353 fix panicing with long link names 2020-06-17 17:27:05 +03:00
Joonas Koivunen
6a96593866 refactor: make Walker easier to use
turns out the internal structure almost supported this already, but
`self.current` needed to be done an Option.
2020-06-17 17:27:05 +03:00
Joonas Koivunen
e25a23380c refactor: split tar helper to own module 2020-06-17 17:27:05 +03:00
Joonas Koivunen
f71c212edd chore: fmt and clippy warnings 2020-06-17 17:27:05 +03:00
Joonas Koivunen
948eeff48f refactor: rename FileMetadata to ipfs_unixfs::Metadata 2020-06-17 17:27:05 +03:00
Joonas Koivunen
9b18427e16 refactor: move ipfs_unixfs::dir::walk to top level 2020-06-17 17:27:05 +03:00
Joonas Koivunen
d61b0424d8 fix: use errors instead of unwraps in /get as well 2020-06-17 17:27:05 +03:00
Joonas Koivunen
7feededa81 fix: refs must use new ipfspath.follow_dagpb_data 2020-06-17 17:27:05 +03:00