18 Commits

Author SHA1 Message Date
Joonas Koivunen
fbe90ab8ec feat: timeout on block/get 2020-07-09 14:15:44 +03:00
Joonas Koivunen
0a9ee274ca temp: upgrade to mpart-async with fix 2020-07-08 13:52:01 +03:00
Joonas Koivunen
76486f8ecb fix: rewrite block/put to support streaming multiparts
streaming multiparts are something we dont yet support, which is keeping
us back in an earlier version of js-ipfs-http-client for conformance
tests.
2020-07-08 13:52:01 +03:00
Joonas Koivunen
d1cd15b4ec
Merge pull request #193 from eqlabs/fix_use_tempfile
fix test using non-unique tempfile
2020-06-18 01:16:08 +03:00
Joonas Koivunen
3ba5ddf1a3 fix test using non-unique tempfile 2020-06-18 00:55:11 +03:00
Joonas Koivunen
27085b3095 chore: update hyper and warp to latest 2020-06-17 23:07:39 +03:00
Joonas Koivunen
6eb276eb28 add: test /get for long and file names 2020-06-17 17:27:05 +03:00
Joonas Koivunen
6671e74113 feat: /get http endpoint
does not at all pass any tests. we need to add a root path param, which
will be a string version of the requested Cid for the http api. Also,
with this, it becomes possible to hide everything (files, symlinks)
under a single walk api.
2020-06-17 17:26:39 +03:00
Joonas Koivunen
79985e4140
Add refs endpoint (#147)
A lot of code but going back and forth as well.

* fix: request logger name for good

* wip: initial broilerplate

* feat: ipfspath parsing and walking

* refactor: rewrite to walk the path before streaming

* fix: handling 0 for ipld link

* wip: get first tests to pass

still need to figure out the Send + Sync + 'static bound, issue is with
Sync, which might come from the refs used, get_block(&cid)..

this required some unsafety to be able to present the Stream as Send +
Sync + 'static as required by hyper. not 100% sure if that pin_project
is ok to do, but the only interfaces exposed are callable as futures so
it shouldn't be too bad.

* feat: format string support
* add: manual RefsOptions deserialization to have multiple `arg`

* fix: resolve multiple paths

I had not originally noticed that we need to do the walk from multiple
paths.

* add: dag-pb specific ipfspath walking

* add: make IpfsPath better debuggable

* wip: adjust logging

* add: yield ipld parsing failed err's

* add: "specialize" ipld_links for dag-pb

* feat: support visiting unique nodes

* fix: instead of stopping, filter out too deep elements

also, this should be done when expanding the nodes, not at this stage.

* refactor: reorg and document

originally it was easier to write everything to the same file, but it
gets difficult to read after 1000 lines. while reorganising stuff into
more tight modules, some direct calls were changed not to be needed,
like RefsOptions creating the EdgeFormatter; the EdgeFormatter is not
created in the refs function itself.

chopped off parts:

 * RefsOptions stuff to refs/options.rs
 * EdgeFormatter and the string format to refs/format.rs
 * Unshared to refs/unshared.rs
 * HandledErr, StreamResponse to refs/support.rs

* refactor: minor cleanup and docs

* fix: refs tests, enable unique test

* fix: use specific rust-ipld version with cbor fix

the situation is not good, as it would make sense to re-export libipld
from ipfs as parts are already re-exported but with strange names.

* refactor: allow dead_code StringError::new

* refactor: use the Edge for refs-local as well

* chore: cargo fmt and reorder tests

* doc: replace fixme with confirmed

* add: port tests for ipfspath

* fix: unignore good_but_unsupported test

* add: test cases for slash dedup per discussion

* add: walk dag-cbor tests for IpfsPath

* add: dag-pb path walking test

* add: tests for walking over a cbor link

* add: test case for dapb link listing

* fix: refs_local test, move all tests under mod test

* fix: remove unwrap from refs-local

kept the return value as a stream. it'll at least save us the trouble of
concatenating all of the values into a big vector.
2020-04-16 16:39:48 +03:00
Mark Robert Henderson
63f81b605a Update Cargo.toml 2020-04-14 08:04:06 -04:00
David Craven
028b8074c3 wip dag put 2020-04-14 08:04:06 -04:00
Joonas Koivunen
72ff94cecf wip salvage
This is originally from the non-merging branch by David in
https://github.com/ipfs-rust/rust-ipfs/pull/126.

Co-authored-by: David Craven <david@craven.ch>
2020-04-02 13:04:00 +03:00
Joonas Koivunen
6cbf679644
Add pubsub http (#120)
* add: pubsub http api
* add: get shoveling and subscription complications going

currently the missing piece is seeing locally sent messages in the
subscribed streams, then maybe creating a more complicated API for
subscribing which will require a single task for shoveling into the
responses.

* chore: lower timeout to pass the tests

Should probably have another drop sender here.

* fix: close stream on first error

tokio::sync::broadcast will continue after telling how many items were
lost, which seems likely with the small buffer and quickly firing more
than the buffer elements.

* fix: raise the buffer size for subscribe responses
* feat: use custom rust-libp2p to pass tests

the v0.16.2-with-floodsub-opts includes "subscribers see locally
published messages" which is needed for the conformance tests to pass.
this has been PR'd as https://github.com/libp2p/rust-libp2p/pull/1520.

* fix: pubsub test after changing the floodsub impl
2020-03-25 21:42:06 +02:00
Joonas Koivunen
ebcb230d6b refactor: rename rust-ipfs-http to ipfs-http 2020-03-17 11:14:03 +02:00
Joonas Koivunen
8b1abb3333 fix: log warning instead of eprintln 2020-03-17 11:08:33 +02:00
Joonas Koivunen
47b9d02a2a feat: rsa keygen using openssl with compat storage
taking on the openssl dependency is used for:

 - rsa key generation
 - from/to `pkcs#1` and `pkcs#8` transformations

Crate `rsa` was also considered but it does not support exporting keys
in any format at the moment and it's import code looks different from
`ring`s (this could be the case of not importing the "precomputed"
values in `rsa`).

The `pkcs#1` is (hopefully) used with go-ipfs-config/go-libp2p
compatible key storage but rust-libp2p/ring need the key in `pkcs#8` der
format. PEM parsing and PEM to DER was implemented by hand as it looked
like the existingly depended on crates are not compatible as much as
possible.
2020-03-12 16:30:36 +02:00
Joonas Koivunen
c316a1c824 add: build metadata with vergen
this could be moved over to rust-ipfs but I think that people will
prefer to track their own binarys version (which is tracked hopefully in
a vcs) instead of their `rust-ipfs` deps version, which will be in their
`Cargo.lock` file.
2020-03-11 14:02:29 -04:00
Joonas Koivunen
e032d3d4aa
Http v0 for testing (#71)
* add first draft of the http api

* chore note on the difficulties with with_graceful_shutdown

* add the second required output line by js-ipfsd-ctl

* run cargo fmt for different formatting from rls?

* chore note on (stalled) progress

* add remaining cli args boilerplate

what is missing now is using ephemeral addresses with the warp, as
js-ipfsd-ctl doesn't always to shutdown the older one, or then it just
doesn't shut down correctly.

* fix bind to ephemeral localhost port

* add env_logger initialization

* fix faked multiaddr typo

never remember its "ip4" instead of "ipv4"

* fix do GET on every method in /id

for some reason js-ipfsd-ctl will do a POST against /api/v0/id.

* chore run cargo fmt

* Adding placeholder NOT_IMPLEMENTED endpiints

* rustfmt

* Feedback from PR

* Update http/Cargo.toml

Co-Authored-By: Joonas Koivunen <joonas@equilibrium.co>

Co-authored-by: Mark Robert Henderson <henderson.mark@gmail.com>
2020-03-05 21:38:44 +02:00