summaryrefslogtreecommitdiff
path: root/guix/http-client.scm
AgeCommit message (Collapse)Author
2022-05-22http-client: 'http-fetch/cached' converts strings to URIs.Philip McGrath
* guix/http-client.scm (http-fetch/cached): Use 'string->uri' if URI is a string, as with 'http-fetch'. Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-05-22http-client: Accept '#:headers' in 'http-fetched/cached'.Philip McGrath
Callers can supply alternative headers as with 'http-fetch'. * guix/http-client.scm (http-fetch/cached): Add '#:headers' argument. Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-04-11http-client: Fix redirection.Attila Lendvai
* guix/http-client.scm (http-fetch): Use the right uri variable in case of redirection. Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-03-06http-client: Correctly handle redirects when #:keep-alive? #t.Ludovic Courtès
Previously PORT would be closed unconditionally, which broke redirects when #:keep-alive? #t is given. * guix/http-client.scm (http-fetch): Make 'port' a parameter of 'loop'. Upon 3xx responses, do not close PORT is KEEP-ALIVE? is true, but consume RESP's body. Add second argument to 'loop'.
2022-03-06http-client: Add response headers to '&http-get-error'.Ludovic Courtès
* guix/http-client.scm (&http-get-error)[headers]: New field. (http-fetch): Initialize the 'headers' field.
2021-04-24http-client: Remove exception mishandling in 'http-multiple-get'.Ludovic Courtès
Partly fixes <https://bugs.gnu.org/47867>. Reported by Florian Pelz <pelzflorian@pelzflorian.de>. The non-tail recursive call to 'connect' could cause requests to be processed twice, with 'p' possibly closed the second time. Regression introduced in 205833b72c5517915a47a50dbe28e7024dc74e57 and carried over in 45fce38fb0b6c6796906149ade145b8d3594c1c6. * guix/http-client.scm (http-multiple-get): Remove call to 'close-port' and recursive call to 'connect' when the 'write-request' block returns #f.
2021-04-24http-client, substitute: Gracefully handle GnuTLS EAGAIN/EINTR.Ludovic Courtès
Partly fixes <https://bugs.gnu.org/47867>. Reported by Florian Pelz <pelzflorian@pelzflorian.de>. In GnuTLS up to 3.7.1 included, GNUTLS_E_AGAIN and GNUTLS_E_INTERRUPTED are not handled by 'write_to_session_record_port' and could be thrown at the caller. This patch works around that by dropping connections altogether and restarting when this happens. * guix/http-client.scm (false-if-networking-error): Swallow ERROR/AGAIN and ERROR/INTERRUPTED. * guix/scripts/substitute.scm (call-with-cached-connection): Likewise.
2021-03-28http-client: Don't drop failed query when reconnecting.Ludovic Courtès
Reported by Christopher Baines <mail@cbaines.net>. * guix/http-client.scm (http-multiple-get): Change 2nd argument to 'drop' to PROCESSED when (false-if-networking-error ...) returns #f.
2021-03-27http-client: 'http-multiple-get' is tail-recursive again.Ludovic Courtès
Fixes <https://bugs.gnu.org/47283>. Commit 205833b72c5517915a47a50dbe28e7024dc74e57 made 'http-multiple-get' non-tail-recursive. Each recursive call would install an exception handler. As the number of iterations grows beyond 1,000, quadratic complexity of 'raise-exception' would show and we'd spend most of our time there. * guix/http-client.scm (false-if-networking-error): New macro. (http-multiple-get): Use it around 'write-request' and 'put-bytevector' calls, and around 'read-response' call, in lieu of the inline 'catch' forms.
2021-03-17http-client: 'http-fetch' and 'http-fetch/cached' accept #:log-port.Ludovic Courtès
* guix/http-client.scm (http-fetch, http-fetch/cached): Add #:log-port and honor it.
2021-02-22http-client: Accept #:open-connection in http-fetch.Christopher Baines
So that an alternative procedure can be passed in, perhaps to perform connection caching. * guix/http-client.scm (http-fetch): Add an #:open-connection keyword argument.
2021-02-22http-client: Add error handling to http-multiple-get.Christopher Baines
Making sure to close the port if it looks to be unusable. This closing of the port will allow for caching connections, without caching broken connections, as the cache can avoid handing out closed ports. * guix/http-client.scm (http-multiple-get): Try to catch exceptions that happen if the port is unusable, this is a adaptation of code within the (guix scripts substitute) module.
2021-02-22guix: Move http-multiple-get to (guix http-client).Christopher Baines
From (guix scripts substitute). This will make it easier to reuse this code. * guix/scripts/substitute.scm (http-multiple-get): Remove, and move to… * guix/http-client.scm (http-multiple-get): …here.
2020-12-08substitute: Cache and reuse connections while substituting.Ludovic Courtès
That way, when fetching a series of substitutes from the same server(s), the connection is reused instead of being closed/opened for each substitutes, which saves on network round trips and TLS handshakes. * guix/http-client.scm (http-fetch): Add #:keep-alive? and honor it. * guix/progress.scm (progress-report-port): Add #:close? parameter and honor it. * guix/scripts/substitute.scm (at-most): Return the tail as a second value. (fetch): Add #:port and #:keep-alive? and honor them. (%max-cached-connections): New variable. (open-connection-for-uri/cached, call-with-cached-connection): New procedures. (with-cached-connection): New macro. (process-substitution): Wrap 'fetch' call in 'with-cached-connection'. Pass #:close? to 'progress-report-port'.
2020-10-12http-client: 'http-fetch' and 'http-fetch/cached' accept #:timeout.Ludovic Courtès
* guix/http-client.scm (http-fetch): Add #:timeout and pass it to 'guix:open-connection-for-uri'. (http-fetch/cached): Add #:timeout parameter and pass it to 'http-fetch'.
2020-01-03download: Do not leak file descriptors on TLS ports.Ludovic Courtès
Fixes <https://bugs.gnu.org/20145>. * guix/build/download.scm (%tls-ports, register-tls-record-port): Remove. (tls-wrap): Remove call to 'register-tls-record-port'. Return a custom binary input/output port instead. This is a backport of what Guile 2.2's (web client) module has been doing. (close-connection): Define as an alias for 'close-port'. * guix/http-client.scm (http-fetch): Remove #:keep-alive? parameter, which was ignored and unused. Pass #:keep-alive? #f to 'http-get'. * guix/lint.scm (probe-uri): Use 'close-port' instead of 'close-connection'. * guix/scripts/substitute.scm (http-multiple-get): Likewise.
2019-01-09Remove most uses of the _IO*F constants.Ludovic Courtès
These constants, for use with 'setvbuf', were deprecated in Guile 2.2 and disappeared in Guile 3.0. Here we keep these constants in build-side code where removing them is not feasible. * guix/build/download-nar.scm (download-nar): Adjust 'setvbuf' calls to the Guile 2.2+ API. * guix/build/download.scm (open-socket-for-uri): Likewise. (open-connection-for-uri, url-fetch): Likewise. * guix/build/make-bootstrap.scm (make-stripped-libc): Likewise. * guix/build/union.scm (setvbuf) [guile-2.0]: New conditional wrapper. (union-build): Adjust to new API. * guix/ftp-client.scm (ftp-open, ftp-list, ftp-retr): Likewise. * guix/http-client.scm (http-fetch): Likewise. * guix/inferior.scm (proxy): Likewise. * guix/scripts/substitute.scm (fetch, http-multiple-get): Likewise. * guix/self.scm (compiled-modules): Likewise. * guix/ssh.scm (remote-daemon-channel, store-import-channel) (store-export-channel): Likewise. * guix/ui.scm (initialize-guix): Likewise. * tests/publish.scm (http-get-port): Likewise. * guix/store.scm (%newlines): Adjust comment.
2018-09-04Switch to Guile-Gcrypt.Ludovic Courtès
This removes (guix hash) and (guix pk-crypto), which now live as part of Guile-Gcrypt (version 0.1.0.) * guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm, tests/hash.scm, tests/pk-crypto.scm: Remove. * configure.ac: Test for Guile-Gcrypt. Remove LIBGCRYPT and LIBGCRYPT_LIBDIR assignments. * m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove. * README: Add Guile-Gcrypt to the dependencies; move libgcrypt as "required unless --disable-daemon". * doc/guix.texi (Requirements): Likewise. * gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm, guix/git.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm, guix/import/gnu.scm, guix/import/hackage.scm, guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm, guix/pki.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/download.scm, guix/scripts/hash.scm, guix/scripts/pack.scm, guix/scripts/publish.scm, guix/scripts/refresh.scm, guix/scripts/substitute.scm, guix/store.scm, guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm, tests/builders.scm, tests/challenge.scm, tests/cpan.scm, tests/crate.scm, tests/derivations.scm, tests/gem.scm, tests/nar.scm, tests/opam.scm, tests/pki.scm, tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm, tests/store.scm, tests/substitute.scm: Adjust imports. * gnu/system/vm.scm: Likewise. (guile-sqlite3&co): Rename to... (gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT. (expression->derivation-in-linux-vm)[config]: Remove. (iso9660-image)[config]: Remove. (qemu-image)[config]: Remove. (system-docker-image)[config]: Remove. * guix/scripts/pack.scm: Adjust imports. (guile-sqlite3&co): Rename to... (gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT. (self-contained-tarball)[build]: Call 'make-config.scm' without #:libgcrypt argument. (squashfs-image)[libgcrypt]: Remove. [build]: Call 'make-config.scm' without #:libgcrypt. (docker-image)[config, json]: Remove. [build]: Add GUILE-GCRYPT to the extensions Remove (guix config) from the imported modules. * guix/self.scm (specification->package): Remove "libgcrypt", add "guile-gcrypt". (compiled-guix): Remove #:libgcrypt. [guile-gcrypt]: New variable. [dependencies]: Add it. [*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call. Add #:extensions. [*config*]: Remove #:libgcrypt from 'make-config.scm' call. (%dependency-variables): Remove %libgcrypt. (make-config.scm): Remove #:libgcrypt. * build-aux/build-self.scm (guile-gcrypt): New variable. (make-config.scm): Remove #:libgcrypt. (build-program)[fake-gcrypt-hash]: New variable. Add (gcrypt hash) to the imported modules. Adjust load path assignments. * gnu/packages/package-management.scm (guix)[propagated-inputs]: Add GUILE-GCRYPT. [arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search path.
2018-05-26http-client: Send redirection messages to stderr.Marius Bakke
* guix/http-client.scm (http-fetch): Use CURRENT-ERROR-PORT instead of default output.
2018-02-26build: Require Guile >= 2.0.13.Ludovic Courtès
* README, configure.ac, doc/guix.texi (Requirements): Increase minimum Guile version from 2.0.9 to 2.0.13. * config-daemon.ac: Remove use of 'GUIX_CHECK_UNBUFFERED_CBIP'. * m4/guix.m4 (GUIX_CHECK_UNBUFFERED_CBIP): Remove. * guix/build/download.scm (current-http-proxy): Remove. * guix/build/syscalls.scm (%libc-errno-pointer, errno): Remove. (syscall->procedure): Use #:return-errno unconditionally. * guix/hash.scm (open-sha256-input-port)[unbuffered]: Remove outdated comment. * guix/http-client.scm (when-guile<=2.0.5-or-otherwise-broken): Remove. <top level>: Remove 'when-guile<=2.0.5-or-otherwise-broken' block. * guix/scripts/substitute.scm (fetch): Remove 'guile-version>?' conditional. * tests/hash.scm (supports-unbuffered-cbip?): Remove. <top level>: Remove 'test-skip' call.
2017-11-16cve: Use 'http-fetch/cached' instead of having custom caching.Ludovic Courtès
That way CVE fetching benefits from 'If-Modified-Since' handling. * guix/http-client.scm (http-fetch/cached): Add #:write-cache and #:cache-miss parameters and honor them. * guix/cve.scm (%current-year-ttl, %past-year-ttl): Reduce. (call-with-cve-port): Remove. (write-cache): New procedure. (fetch-vulnerabilities): Rewrite in terms of 'http-fetch/cached'.
2017-09-26http-client: Reset cache TTL upon 304 "Not Modified" responses.Ludovic Courtès
* guix/http-client.scm (http-fetch/cached)[update-cache]: Add call to 'utime' in the 304 case.
2017-09-19http-client: 'http-client/cached' uses 'If-Modified-Since'.Ludovic Courtès
* guix/http-client.scm (http-fetch/cached)[update-cache]: Add 'cache-port' parameter. Check its mtime and compute 'if-modified-since' header accordingly. Guard 'http-get-error?' and honor 304. Adjust callers of 'update-cache'. * guix/gnu-maintenance.scm (ftp.gnu.org-files): Set #:ttl to 15m.
2017-09-05Handle the same HTTP redirects everywhere.Tobias Geerinckx-Rice
* guix/build/download.scm (http-fetch): Complete the hard-coded list of HTTP redirection status codes. * guix/http-client.scm (http-fetch): Likewise. * guix/scripts/lint.scm (probe-uri): Likewise.
2017-05-03ui: Rename '_' to 'G_'.Ludovic Courtès
This avoids collisions with '_' when the latter is used as a 'match' pattern for instance. See <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>. * guix/ui.scm: Rename '_' to 'G_'. * po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly. * build-aux/compile-all.scm (warnings): Remove 'format'. * gnu/packages.scm, gnu/services.scm, gnu/services/shepherd.scm, gnu/system.scm, gnu/system/shadow.scm, guix/gnupg.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/elpa.scm, guix/import/pypi.scm, guix/nar.scm, guix/scripts.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/build.scm, guix/scripts/challenge.scm, guix/scripts/container.scm, guix/scripts/container/exec.scm, guix/scripts/copy.scm, guix/scripts/download.scm, guix/scripts/edit.scm, guix/scripts/environment.scm, guix/scripts/gc.scm, guix/scripts/graph.scm, guix/scripts/hash.scm, guix/scripts/import.scm, guix/scripts/import/cpan.scm, guix/scripts/import/cran.scm, guix/scripts/import/crate.scm, guix/scripts/import/elpa.scm, guix/scripts/import/gem.scm, guix/scripts/import/gnu.scm, guix/scripts/import/hackage.scm, guix/scripts/import/nix.scm, guix/scripts/import/pypi.scm, guix/scripts/import/stackage.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/scripts/pack.scm, guix/scripts/package.scm, guix/scripts/perform-download.scm, guix/scripts/publish.scm, guix/scripts/pull.scm, guix/scripts/refresh.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/scripts/system.scm, guix/ssh.scm, guix/upstream.scm: Use 'G_' instead of '_'. Most of this change was obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`".
2017-03-18http-client: Avoid name clash with 'open-connection-for-uri' in 2.2.0.Ludovic Courtès
* guix/build/download.scm (open-connection-for-uri): Add note about same-named binding in Guile 2.2.0. * guix/http-client.scm: Use 'guix:open-connection-for-uri' for the procedure coming from (guix build download). * guix/scripts/lint.scm: Likewise. * guix/scripts/substitute.scm: Likewise.
2017-03-18build: Require Guile >= 2.0.9.Ludovic Courtès
* configure.ac: Bump requirement to 2.0.9. * doc/guix.texi (Requirements): Adjust accordingly. * README (Requirements): Likewise. * build-aux/download.scm: Remove workaround for <http://bugs.gnu.org/13095>. * guix/build/download.scm: Likewise. (http-fetch)[post-2.0.7?]: Remove. Remove conditional code for not POST-2.0.7?. * guix/http-client.scm: Remove workaround for <http://bugs.gnu.org/13095>. (http-fetch)[post-2.0.7?]: Remove. Remove conditional code for not POST-2.0.7?. * guix/serialization.scm (read-latin1-string): Remove mention of 2.0.9. * tests/nar.scm: Use (ice-9 control). (let/ec): Remove.
2017-01-13http-client: Provide 'User-Agent' header by default.Ludovic Courtès
* guix/http-client.scm (http-fetch): Add #:headers parameter and honor it. Rename 'auth-header' to 'headers'. * guix/import/github.scm (json-fetch*): Add comment about required User-Agent.
2017-01-10http-client: Improve error reporting.Ludovic Courtès
* guix/http-client.scm (http-fetch): Change message in &message condition to include URI, CODE, and the reason phrase.
2016-11-12http-client: Add #:verify-certificate? to 'http-fetch'.Ludovic Courtès
* guix/http-client.scm (http-fetch): Add #:verify-certificate? parameter and pass it to 'open-connection-for-uri'.
2016-10-19Use (ice-9 binary-ports) instead of (rnrs io ports).Ludovic Courtès
This reduces the closure of (guix ui) from 123 to 106 modules. * guix/derivations.scm: Use (ice-9 binary-ports) instead of (rnrs io ports). (map-derivation)[substitute-file]: Use 'read-string' instead of 'get-string-all'. * guix/ftp-client.scm: Likewise. * guix/hash.scm: Likewise. * guix/http-client.scm: Likewise. * guix/pki.scm (ensure-acl, current-acl): Likewise. * guix/scripts/archive.scm (authorize-key)[read-key]: Likewise. * guix/scripts/authenticate.scm (read-canonical-sexp) (read-hash-data): Likewise. * guix/scripts/download.scm: Likewise. * guix/scripts/offload.scm (register-gc-root, remove-gc-roots) (send-files): Likewise. * guix/scripts/publish.scm (lazy-read-file-sexp): Likewise. * guix/scripts/refresh.scm: Likewise. * guix/scripts/substitute.scm (check-acl-initialized): Likewise. * guix/serialization.scm (read-maybe-utf8-string): Likewise. * guix/scripts/hash.scm (guix-hash): Use 'force-output' instead of 'flush-output-port'. * guix/store.scm (process-stderr): Likewise. * guix/tests.scm: Likewise. * guix/utils.scm: Use (ice-9 binary-ports) and autoload (rnrs io ports) for 'make-custom-binary-input-port'.
2016-03-16http-client: No 'setvbuf' for non-file ports.Ludovic Courtès
* guix/http-client.scm (http-fetch): Do not call 'setvbuf' on non-file ports.
2016-03-14http-client: Add #:keep-alive? parameter.Ludovic Courtès
* guix/http-client.scm (http-fetch): Add #:keep-alive? parameter and pass it to 'http-get' or 'http-get*'.
2016-02-25http-client: 'http-client/cached' uses unique cache file names.Ludovic Courtès
* guix/http-client.scm (cache-file-for-uri): New procedure. (http-fetch/cached): Use it. Remove 'directory' variable. [update-cache]: Make the 'dirname' of FILE.
2016-02-25http-client: 'http-fetch/cached' updates the cache atomically.Ludovic Courtès
* guix/http-client.scm (http-fetch/cached)[update-cache]: Use 'with-atomic-file-output' instead of 'call-with-output-file'.
2016-01-06http-client: Work around <http://bugs.gnu.org/22273>.Ludovic Courtès
* guix/http-client.scm (read-header-line): New procedure. Use it.
2016-01-06http-client: Fix erroneous comment.Ludovic Courtès
* guix/http-client.scm: Fix Guile version number in comment for Guile commit 5a10e41.
2015-12-21http-client: Support basic authentication.Ricardo Wurmus
* guix/http-client.scm (http-fetch): Add Authorization header to request when the URI contains userinfo.
2015-11-29http-client: 'http-fetch' converts strings to URIs.Ludovic Courtès
* guix/http-client.scm (http-fetch): Use 'string->uri' if URI is a string. Fixes a regression introduced in 8a5063f.
2015-11-26http-client: 'http-fetch' and 'http-fetch/cached' support HTTPS.Ludovic Courtès
* guix/http-client.scm (http-fetch): Use 'open-connection-for-uri', to support HTTPS.
2015-10-21http-client: '%http-cache-ttl' is really a parameter.Ludovic Courtès
Fixes a typo in commit 739ab68 that made it a procedure returning a parameter. * guix/http-client.scm (%http-cache-ttl): Turn into a parameter.
2015-10-17http-client: Add 'http-fetch/cached'.Ludovic Courtès
* guix/utils.scm (cache-directory): New procedure. * guix/http-client.scm (%http-cache-ttl): New variable. (http-fetch/cached): New procedure.
2015-09-26http-client: Backport Guile fix for 'read-chunk-header'.Ludovic Courtès
Fixes the wrong-type-arg exception initially reported at <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19976#5> by Ricardo Wurmus <rekado@elephly.net>. * guix/http-client.scm (read-chunk-header): Backport Guile commit 53b8d5f.
2015-09-10http-client: Backport delimited input port fix from Guile.Ludovic Courtès
* guix/http-client.scm (make-delimited-input-port): Backport Guile commit 5a10e41. Monkey-patch 'make-delimited-input-port' for any Guile <= 2.0.11.
2015-09-10http-client: Remove redundant import clause.Ludovic Courtès
* guix/http-client.scm: Remove redundant #:use-module (guix utils).
2015-05-10http-client: Remove monkey patching for 2.0.5.Ludovic Courtès
* guix/http-client.scm (read-response-body*): Remove. (http-fetch): Remove hacks for 2.0.5.
2015-05-06download: Work around Guile small-receive-buffer bug.Ludovic Courtès
Previously, code using directly (guix build download) was still affected by <http://bugs.gnu.org/15368>. This includes source derivations, the 'guix download' command, and (guix gnu-maintenance). 'guix substitute' was unaffected since it used (guix http-client), which already had the fix. * guix/http-client.scm (open-socket-for-uri): Remove. (http-fetch): Remove #:buffered? argument to 'open-socket-for-uri'; use 'setvbuf' instead. * guix/scripts/substitute.scm (fetch): Likewise. * guix/build/download.scm (open-socket-for-uri): New procedure, taken from guix/http-client.scm, but without the #:buffered? parameter.
2015-04-08http-client: Add workaround for HTTP pipelining on Guile <= 2.0.9.Ludovic Courtès
Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>. * guix/http-client.scm (make-delimited-input-port): New procedure. Install it in (web response) for Guile <= 2.0.9.
2015-03-03http-client: Monkey-patch 'make-chunked-input-port' on Guile <= 2.0.11.Ludovic Courtès
Fixes <http://bugs.gnu.org/19976>. * guix/http-client.scm (when-guile<=2.0.5): Rename to... (when-guile<=2.0.5-or-otherwise-broken): ... this. (%web-http): New variable. Monkey-patch 'make-chunked-input-port' when %WEB-HTTP defines 'read-chunk-body'.
2015-03-03http-client: Update backport of chunked encoding support to Guile 2.0.5.Ludovic Courtès
* guix/http-client.scm (read-chunk, read-chunk-body) [when-guile<=2.0.5]: Remove. (make-chunked-input-port) [when-guile<=2.0.5]: Update to Guile commit 00d3ecf2.
generated by cgit v1.2.3 (git 2.44.0) at 2024-06-29 15:49:33 +0000