shaba/awx
1
0
Fork 0
mirror of https://github.com/ansible/awx.git synced 2024-10-30 22:21:13 +03:00
Code Releases Activity

Fixing the Redis vendor.

Browse Source
This commit is contained in:
Luke Sneeringer 2014-11-03 15:13:06 -06:00
parent f69969f1e0
commit c3920ebf4b
26 changed files with 0 additions and 4858 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

426
awx/lib/site-packages/redis/CHANGES
View File

@ -1,426 +0,0 @@
* 2.10.3
* Fixed a bug with the bytearray support introduced in 2.10.2. Thanks
Josh Owen.
* 2.10.2
* Added support for Hiredis's new bytearray support. Thanks
https://github.com/tzickel
* POSSIBLE BACKWARDS INCOMPATBLE CHANGE: Fixed a possible race condition
when multiple threads share the same Lock instance with a timeout. Lock
tokens are now stored in thread local storage by default. If you have
code that acquires a lock in one thread and passes that lock instance to
another thread to release it, you need to disable thread local storage.
Refer to the doc strings on the Lock class about the thread_local
argument information.
* Fixed a regression in from_url where "charset" and "errors" weren't
valid options. "encoding" and "encoding_errors" are still accepted
and preferred.
* The "charset" and "errors" options have been deprecated. Passing
either to StrictRedis.__init__ or from_url will still work but will
also emit a DeprecationWarning. Instead use the "encoding" and
"encoding_errors" options.
* Fixed a compatability bug with Python 3 when the server closes a
connection.
* Added BITPOS command. Thanks https://github.com/jettify.
* Fixed a bug when attempting to send large values to Redis in a Pipeline.
* 2.10.1
* Fixed a bug where Sentinel connections to a server that's no longer a
master and receives a READONLY error will disconnect and reconnect to
the master.
* 2.10.0
* Discontinuted support for Python 2.5. Upgrade. You'll be happier.
* The HiRedis parser will now properly raise ConnectionErrors.
* Completely refactored PubSub support. Fixes all known PubSub bugs and
adds a bunch of new features. Docs can be found in the README under the
new "Publish / Subscribe" section.
* Added the new HyperLogLog commanads (PFADD, PFCOUNT, PFMERGE). Thanks
Pepijn de Vos and Vincent Ohprecio.
* Updated TTL and PTTL commands with Redis 2.8+ semantics. Thanks Markus
Kaiserswerth.
* *SCAN commands now return a long (int on Python3) cursor value rather
than the string representation. This might be slightly backwards
incompatible in code using *SCAN commands loops such as
"while cursor != '0':".
* Added extra *SCAN commands that return iterators instead of the normal
[cursor, data] type. Use scan_iter, hscan_iter, sscan_iter, and
zscan_iter for iterators. Thanks Mathieu Longtin.
* Added support for SLOWLOG commands. Thanks Rick van Hattem.
* Added lexicographical commands ZRANGEBYLEX, ZREMRANGEBYLEX, and ZLEXCOUNT
for sorted sets.
* Connection objects now support an optional argument, socket_read_size,
indicating how much data to read during each socket.recv() call. After
benchmarking, increased the default size to 64k, which dramatically
improves performance when fetching large values, such as many results
in a pipeline or a large (>1MB) string value.
* Improved the pack_command and send_packed_command functions to increase
performance when sending large (>1MB) values.
* Sentinel Connections to master servers now detect when a READONLY error
is encountered and disconnect themselves and all other active connections
to the same master so that the new master can be discovered.
* Fixed Sentinel state parsing on Python 3.
* Added support for SENTINEL MONITOR, SENTINEL REMOVE, and SENTINEL SET
commands. Thanks Greg Murphy.
* INFO ouput that doesn't follow the "key:value" format will now be
appended to a key named "__raw__" in the INFO dictionary. Thanks Pedro
Larroy.
* The "vagrant" directory contains a complete vagrant environment for
redis-py developers. The environment runs a Redis master, a Redis slave,
and 3 Sentinels. Future iterations of the test sutie will incorporate
more integration style tests, ensuring things like failover happen
correctly.
* It's now possible to create connection pool instances from a URL.
StrictRedis.from_url() now uses this feature to create a connection pool
instance and use that when creating a new client instance. Thanks
https://github.com/chillipino
* When creating client instances or connection pool instances from an URL,
it's now possible to pass additional options to the connection pool with
querystring arguments.
* Fixed a bug where some encodings (like utf-16) were unusable on Python 3
as command names and literals would get encoded.
* Added an SSLConnection class that allows for secure connections through
stunnel or other means. Construct and SSL connection with the sll=True
option on client classes, using the rediss:// scheme from an URL, or
by passing the SSLConnection class to a connection pool's
connection_class argument. Thanks https://github.com/oranagra.
* Added a socket_connect_timeout option to control how long to wait while
establishing a TCP connection before timing out. This lets the client
fail fast when attempting to connect to a downed server while keeping
a more lenient timeout for all other socket operations.
* Added TCP Keep-alive support by passing use the socket_keepalive=True
option. Finer grain control can be achieved using the
socket_keepalive_options option which expects a dictionary with any of
the keys (socket.TCP_KEEPIDLE, socket.TCP_KEEPCNT, socket.TCP_KEEPINTVL)
and integers for values. Thanks Yossi Gottlieb.
* Added a `retry_on_timeout` option that controls how socket.timeout errors
are handled. By default it is set to False and will cause the client to
raise a TimeoutError anytime a socket.timeout is encountered. If
`retry_on_timeout` is set to True, the client will retry a command that
timed out once like other `socket.error`s.
* Completely refactored the Lock system. There is now a LuaLock class
that's used when the Redis server is capable of running Lua scripts along
with a fallback class for Redis servers < 2.6. The new locks fix several
subtle race consider that the old lock could face. In additional, a
new method, "extend" is available on lock instances that all a lock
owner to extend the amount of time they have the lock for. Thanks to
Eli Finkelshteyn and https://github.com/chillipino for contributions.
* 2.9.1
* IPv6 support. Thanks https://github.com/amashinchi
* 2.9.0
* Performance improvement for packing commands when using the PythonParser.
Thanks Guillaume Viot.
* Executing an empty pipeline transaction no longer sends MULTI/EXEC to
the server. Thanks EliFinkelshteyn.
* Errors when authenticating (incorrect password) and selecting a database
now close the socket.
* Full Sentinel support thanks to Vitja Makarov. Thanks!
* Better repr support for client and connection pool instances. Thanks
Mark Roberts.
* Error messages that the server sends to the client are now included
in the client error message. Thanks Sangjin Lim.
* Added the SCAN, SSCAN, HSCAN, and ZSCAN commands. Thanks Jingchao Hu.
* ResponseErrors generated by pipeline execution provide addition context
including the position of the command in the pipeline and the actual
command text generated the error.
* ConnectionPools now play nicer in threaded environments that fork. Thanks
Christian Joergensen.
* 2.8.0
* redis-py should play better with gevent when a gevent Timeout is raised.
Thanks leifkb.
* Added SENTINEL command. Thanks Anna Janackova.
* Fixed a bug where pipelines could potentially corrupt a connection
if the MULTI command generated a ResponseError. Thanks EliFinkelshteyn
for the report.
* Connections now call socket.shutdown() prior to socket.close() to
ensure communication ends immediately per the note at
http://docs.python.org/2/library/socket.html#socket.socket.close
Thanks to David Martin for pointing this out.
* Lock checks are now based on floats rather than ints. Thanks
Vitja Makarov.
* 2.7.6
* Added CONFIG RESETSTAT command. Thanks Yossi Gottlieb.
* Fixed a bug introduced in 2.7.3 that caused issues with script objects
and pipelines. Thanks Carpentier Pierre-Francois.
* Converted redis-py's test suite to use the awesome py.test library.
* Fixed a bug introduced in 2.7.5 that prevented a ConnectionError from
being raised when the Redis server is LOADING data.
* Added a BusyLoadingError exception that's raised when the Redis server
is starting up and not accepting commands yet. BusyLoadingError
subclasses ConnectionError, which this state previously returned.
Thanks Yossi Gottlieb.
* 2.7.5
* DEL, HDEL and ZREM commands now return the numbers of keys deleted
instead of just True/False.
* from_url now supports URIs with a port number. Thanks Aaron Westendorf.
* 2.7.4
* Added missing INCRBY method. Thanks Krzysztof Dorosz.
* SET now accepts the EX, PX, NX and XX options from Redis 2.6.12. These
options will generate errors if these options are used when connected
to a Redis server < 2.6.12. Thanks George Yoshida.
* 2.7.3
* Fixed a bug with BRPOPLPUSH and lists with empty strings.
* All empty except: clauses have been replaced to only catch Exception
subclasses. This prevents a KeyboardInterrupt from triggering exception
handlers. Thanks Lucian Branescu Mihaila.
* All exceptions that are the result of redis server errors now share a
command Exception subclass, ServerError. Thanks Matt Robenolt.
* Prevent DISCARD from being called if MULTI wasn't also called. Thanks
Pete Aykroyd.
* SREM now returns an integer indicating the number of items removed from
the set. Thanks http://github.com/ronniekk.
* Fixed a bug with BGSAVE and BGREWRITEAOF response callbacks with Python3.
Thanks Nathan Wan.
* Added CLIENT GETNAME and CLIENT SETNAME commands.
Thanks http://github.com/bitterb.
* It's now possible to use len() on a pipeline instance to determine the
number of commands that will be executed. Thanks Jon Parise.
* Fixed a bug in INFO's parse routine with floating point numbers. Thanks
Ali Onur Uyar.
* Fixed a bug with BITCOUNT to allow `start` and `end` to both be zero.
Thanks Tim Bart.
* The transaction() method now accepts a boolean keyword argument,
value_from_callable. By default, or if False is passes, the transaction()
method will return the value of the pipelines execution. Otherwise, it
will return whatever func() returns.
* Python3 compatibility fix ensuring we're not already bytes(). Thanks
Salimane Adjao Moustapha.
* Added PSETEX. Thanks YAMAMOTO Takashi.
* Added a BlockingConnectionPool to limit the number of connections that
can be created. Thanks James Arthur.
* SORT now accepts a `groups` option that if specified, will return
tuples of n-length, where n is the number of keys specified in the GET
argument. This allows for convenient row-based iteration. Thanks
Ionuț Arțăriși.
* 2.7.2
* Parse errors are now *always* raised on multi/exec pipelines, regardless
of the `raise_on_error` flag. See
https://groups.google.com/forum/?hl=en&fromgroups=#!topic/redis-db/VUiEFT8U8U0
for more info.
* 2.7.1
* Packaged tests with source code
* 2.7.0
* Added BITOP and BITCOUNT commands. Thanks Mark Tozzi.
* Added the TIME command. Thanks Jason Knight.
* Added support for LUA scripting. Thanks to Angus Peart, Drew Smathers,
Issac Kelly, Louis-Philippe Perron, Sean Bleier, Jeffrey Kaditz, and
Dvir Volk for various patches and contributions to this feature.
* Changed the default error handling in pipelines. By default, the first
error in a pipeline will now be raised. A new parameter to the
pipeline's execute, `raise_on_error`, can be set to False to keep the
old behavior of embeedding the exception instances in the result.
* Fixed a bug with pipelines where parse errors won't corrupt the
socket.
* Added the optional `number` argument to SRANDMEMBER for use with
Redis 2.6+ servers.
* Added PEXPIRE/PEXPIREAT/PTTL commands. Thanks Luper Rouch.
* Added INCRBYFLOAT/HINCRBYFLOAT commands. Thanks Nikita Uvarov.
* High precision floating point values won't lose their precision when
being sent to the Redis server. Thanks Jason Oster and Oleg Pudeyev.
* Added CLIENT LIST/CLIENT KILL commands
* 2.6.2
* `from_url` is now available as a classmethod on client classes. Thanks
Jon Parise for the patch.
* Fixed several encoding errors resulting from the Python 3.x support.
* 2.6.1
* Python 3.x support! Big thanks to Alex Grönholm.
* Fixed a bug in the PythonParser's read_response that could hide an error
from the client (#251).
* 2.6.0
* Changed (p)subscribe and (p)unsubscribe to no longer return messages
indicating the channel was subscribed/unsubscribed to. These messages
are available in the listen() loop instead. This is to prevent the
following scenario:
* Client A is subscribed to "foo"
* Client B publishes message to "foo"
* Client A subscribes to channel "bar" at the same time.
Prior to this change, the subscribe() call would return the published
messages on "foo" rather than the subscription confirmation to "bar".
* Added support for GETRANGE, thanks Jean-Philippe Caruana
* A new setting "decode_responses" specifies whether return values from
Redis commands get decoded automatically using the client's charset
value. Thanks to Frankie Dintino for the patch.
* 2.4.13
* redis.from_url() can take an URL representing a Redis connection string
and return a client object. Thanks Kenneth Reitz for the patch.
* 2.4.12
* ConnectionPool is now fork-safe. Thanks Josiah Carson for the patch.
* 2.4.11
* AuthenticationError will now be correctly raised if an invalid password
is supplied.
* If Hiredis is unavailable, the HiredisParser will raise a RedisError
if selected manually.
* Made the INFO command more tolerant of Redis changes formatting. Fix
for #217.
* 2.4.10
* Buffer reads from socket in the PythonParser. Fix for a Windows-specific
bug (#205).
* Added the OBJECT and DEBUG OBJECT commands.
* Added __del__ methods for classes that hold on to resources that need to
be cleaned up. This should prevent resource leakage when these objects
leave scope due to misuse or unhandled exceptions. Thanks David Wolever
for the suggestion.
* Added the ECHO command for completeness.
* Fixed a bug where attempting to subscribe to a PubSub channel of a Redis
server that's down would blow out the stack. Fixes #179 and #195. Thanks
Ovidiu Predescu for the test case.
* StrictRedis's TTL command now returns a -1 when querying a key with no
expiration. The Redis class continues to return None.
* ZADD and SADD now return integer values indicating the number of items
added. Thanks Homer Strong.
* Renamed the base client class to StrictRedis, replacing ZADD and LREM in
favor of their official argument order. The Redis class is now a subclass
of StrictRedis, implementing the legacy redis-py implementations of ZADD
and LREM. Docs have been updated to suggesting the use of StrictRedis.
* SETEX in StrictRedis is now compliant with official Redis SETEX command.
the name, value, time implementation moved to "Redis" for backwards
compatability.
* 2.4.9
* Removed socket retry logic in Connection. This is the responsbility of
the caller to determine if the command is safe and can be retried. Thanks
David Wolver.
* Added some extra guards around various types of exceptions being raised
when sending or parsing data. Thanks David Wolver and Denis Bilenko.
* 2.4.8
* Imported with_statement from __future__ for Python 2.5 compatability.
* 2.4.7
* Fixed a bug where some connections were not getting released back to the
connection pool after pipeline execution.
* Pipelines can now be used as context managers. This is the preferred way
of use to ensure that connections get cleaned up properly. Thanks
David Wolever.
* Added a convenience method called transaction() on the base Redis class.
This method eliminates much of the boilerplate used when using pipelines
to watch Redis keys. See the documentation for details on usage.
* 2.4.6
* Variadic arguments for SADD, SREM, ZREN, HDEL, LPUSH, and RPUSH. Thanks
Raphaël Vinot.
* (CRITICAL) Fixed an error in the Hiredis parser that occasionally caused
the socket connection to become corrupted and unusable. This became
noticeable once connection pools started to be used.
* ZRANGE, ZREVRANGE, ZRANGEBYSCORE, and ZREVRANGEBYSCORE now take an
additional optional argument, score_cast_func, which is a callable used
to cast the score value in the return type. The default is float.
* Removed the PUBLISH method from the PubSub class. Connections that are
[P]SUBSCRIBEd cannot issue PUBLISH commands, so it doesn't make sense
to have it here.
* Pipelines now contain WATCH and UNWATCH. Calling WATCH or UNWATCH from
the base client class will result in a deprecation warning. After
WATCHing one or more keys, the pipeline will be placed in immediate
execution mode until UNWATCH or MULTI are called. Refer to the new
pipeline docs in the README for more information. Thanks to David Wolever
and Randall Leeds for greatly helping with this.
* 2.4.5
* The PythonParser now works better when reading zero length strings.
* 2.4.4
* Fixed a typo introduced in 2.4.3
* 2.4.3
* Fixed a bug in the UnixDomainSocketConnection caused when trying to
form an error message after a socket error.
* 2.4.2
* Fixed a bug in pipeline that caused an exception while trying to
reconnect after a connection timeout.
* 2.4.1
* Fixed a bug in the PythonParser if disconnect is called before connect.
* 2.4.0
* WARNING: 2.4 contains several backwards incompatible changes.
* Completely refactored Connection objects. Moved much of the Redis
protocol packing for requests here, and eliminated the nasty dependencies
it had on the client to do AUTH and SELECT commands on connect.
* Connection objects now have a parser attribute. Parsers are responsible
for reading data Redis sends. Two parsers ship with redis-py: a
PythonParser and the HiRedis parser. redis-py will automatically use the
HiRedis parser if you have the Python hiredis module installed, otherwise
it will fall back to the PythonParser. You can force or the other, or even
an external one by passing the `parser_class` argument to ConnectionPool.
* Added a UnixDomainSocketConnection for users wanting to talk to the Redis
instance running on a local machine only. You can use this connection
by passing it to the `connection_class` argument of the ConnectionPool.
* Connections no longer derive from threading.local. See threading.local
note below.
* ConnectionPool has been comletely refactored. The ConnectionPool now
maintains a list of connections. The redis-py client only hangs on to
a ConnectionPool instance, calling get_connection() anytime it needs to
send a command. When get_connection() is called, the command name and
any keys involved in the command are passed as arguments. Subclasses of
ConnectionPool could use this information to identify the shard the keys
belong to and return a connection to it. ConnectionPool also implements
disconnect() to force all connections in the pool to disconnect from
the Redis server.
* redis-py no longer support the SELECT command. You can still connect to
a specific database by specifing it when instantiating a client instance
or by creating a connection pool. If you need to talk to multiplate
databases within your application, you should use a separate client
instance for each database you want to talk to.
* Completely refactored Publish/Subscribe support. The subscribe and listen
commands are no longer available on the redis-py Client class. Instead,
the `pubsub` method returns an instance of the PubSub class which contains
all publish/subscribe support. Note, you can still PUBLISH from the
redis-py client class if you desire.
* Removed support for all previously deprecated commands or options.
* redis-py no longer uses threading.local in any way. Since the Client
class no longer holds on to a connection, it's no longer needed. You can
now pass client instances between threads, and commands run on those
threads will retrieve an available connection from the pool, use it and
release it. It should now be trivial to use redis-py with eventlet or
greenlet.
* ZADD now accepts pairs of value=score keyword arguments. This should help
resolve the long standing #72. The older value and score arguments have
been deprecated in favor of the keyword argument style.
* Client instances now get their own copy of RESPONSE_CALLBACKS. The new
set_response_callback method adds a user defined callback to the instance.
* Support Jython, fixing #97. Thanks to Adam Vandenberg for the patch.
* Using __getitem__ now properly raises a KeyError when the key is not
found. Thanks Ionuț Arțăriși for the patch.
* Newer Redis versions return a LOADING message for some commands while
the database is loading from disk during server start. This could cause
problems with SELECT. We now force a socket disconnection prior to
raising a ResponseError so subsuquent connections have to reconnect and
re-select the appropriate database. Thanks to Benjamin Anderson for
finding this and fixing.
* 2.2.4
* WARNING: Potential backwards incompatible change - Changed order of
parameters of ZREVRANGEBYSCORE to match those of the actual Redis command.
This is only backwards-incompatible if you were passing max and min via
keyword args. If passing by normal args, nothing in user code should have
to change. Thanks Stéphane Angel for the fix.
* Fixed INFO to properly parse the Redis data correctly for both 2.2.x and
2.3+. Thanks Stéphane Angel for the fix.
* Lock objects now store their timeout value as a float. This allows floats
to be used as timeout values. No changes to existing code required.
* WATCH now supports multiple keys. Thanks Rich Schumacher.
* Broke out some code that was Python 2.4 incompatible. redis-py should
now be useable on 2.4, but this hasn't actually been tested. Thanks
Dan Colish for the patch.
* Optimized some code using izip and islice. Should have a pretty good
speed up on larger data sets. Thanks Dan Colish.
* Better error handling when submitting an empty mapping to HMSET. Thanks
Dan Colish.
* Subscription status is now reset after every (re)connection.
* 2.2.3
* Added support for Hiredis. To use, simply "pip install hiredis" or
"easy_install hiredis". Thanks for Pieter Noordhuis for the hiredis-py
bindings and the patch to redis-py.
* The connection class is chosen based on whether hiredis is installed
or not. To force the use of the PythonConnection, simply create
your own ConnectionPool instance with the connection_class argument
assigned to to PythonConnection class.
* Added missing command ZREVRANGEBYSCORE. Thanks Jay Baird for the patch.
* The INFO command should be parsed correctly on 2.2.x server versions
and is backwards compatible with older versions. Thanks Brett Hoerner.
* 2.2.2
* Fixed a bug in ZREVRANK where retriving the rank of a value not in
the zset would raise an error.
* Fixed a bug in Connection.send where the errno import was getting
overwritten by a local variable.
* Fixed a bug in SLAVEOF when promoting an existing slave to a master.
* Reverted change of download URL back to redis-VERSION.tar.gz. 2.2.1's
change of this actually broke Pypi for Pip installs. Sorry!
* 2.2.1
* Changed archive name to redis-py-VERSION.tar.gz to not conflict
with the Redis server archive.
* 2.2.0
* Implemented SLAVEOF
* Implemented CONFIG as config_get and config_set
* Implemented GETBIT/SETBIT
* Implemented BRPOPLPUSH
* Implemented STRLEN
* Implemented PERSIST
* Implemented SETRANGE

6
awx/lib/site-packages/redis/INSTALL
View File

@ -1,6 +0,0 @@
Please use
python setup.py install
and report errors to Andy McCurdy (sedrik@gmail.com)

22
awx/lib/site-packages/redis/LICENSE
View File

@ -1,22 +0,0 @@
Copyright (c) 2012 Andy McCurdy
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

7
awx/lib/site-packages/redis/MANIFEST.in
View File

@ -1,7 +0,0 @@
include CHANGES
include INSTALL
include LICENSE
include README.rst
exclude __pycache__
recursive-include tests *
recursive-exclude tests *.pyc

696
awx/lib/site-packages/redis/PKG-INFO
View File

@ -1,696 +0,0 @@
Metadata-Version: 1.1
Name: redis
Version: 2.10.3
Summary: Python client for Redis key-value store
Home-page: http://github.com/andymccurdy/redis-py
Author: Andy McCurdy
Author-email: sedrik@gmail.com
License: MIT
Description: redis-py
========
The Python interface to the Redis key-value store.
.. image:: https://secure.travis-ci.org/andymccurdy/redis-py.png?branch=master
:target: http://travis-ci.org/andymccurdy/redis-py
Installation
------------
redis-py requires a running Redis server. See `Redis's quickstart
<http://redis.io/topics/quickstart>`_ for installation instructions.
To install redis-py, simply:
.. code-block:: bash
$ sudo pip install redis
or alternatively (you really should be using pip though):
.. code-block:: bash
$ sudo easy_install redis
or from source:
.. code-block:: bash
$ sudo python setup.py install
Getting Started
---------------
.. code-block:: pycon
>>> import redis
>>> r = redis.StrictRedis(host='localhost', port=6379, db=0)
>>> r.set('foo', 'bar')
True
>>> r.get('foo')
'bar'
API Reference
-------------
The `official Redis command documentation <http://redis.io/commands>`_ does a
great job of explaining each command in detail. redis-py exposes two client
classes that implement these commands. The StrictRedis class attempts to adhere
to the official command syntax. There are a few exceptions:
* **SELECT**: Not implemented. See the explanation in the Thread Safety section
below.
* **DEL**: 'del' is a reserved keyword in the Python syntax. Therefore redis-py
uses 'delete' instead.
* **CONFIG GET|SET**: These are implemented separately as config_get or config_set.
* **MULTI/EXEC**: These are implemented as part of the Pipeline class. The
pipeline is wrapped with the MULTI and EXEC statements by default when it
is executed, which can be disabled by specifying transaction=False.
See more about Pipelines below.
* **SUBSCRIBE/LISTEN**: Similar to pipelines, PubSub is implemented as a separate
class as it places the underlying connection in a state where it can't
execute non-pubsub commands. Calling the pubsub method from the Redis client
will return a PubSub instance where you can subscribe to channels and listen
for messages. You can only call PUBLISH from the Redis client (see
`this comment on issue #151
<https://github.com/andymccurdy/redis-py/issues/151#issuecomment-1545015>`_
for details).
* **SCAN/SSCAN/HSCAN/ZSCAN**: The *SCAN commands are implemented as they
exist in the Redis documentation. In addition, each command has an equivilant
iterator method. These are purely for convenience so the user doesn't have
to keep track of the cursor while iterating. Use the
scan_iter/sscan_iter/hscan_iter/zscan_iter methods for this behavior.
In addition to the changes above, the Redis class, a subclass of StrictRedis,
overrides several other commands to provide backwards compatibility with older
versions of redis-py:
* **LREM**: Order of 'num' and 'value' arguments reversed such that 'num' can
provide a default value of zero.
* **ZADD**: Redis specifies the 'score' argument before 'value'. These were swapped
accidentally when being implemented and not discovered until after people
were already using it. The Redis class expects \*args in the form of:
`name1, score1, name2, score2, ...`
* **SETEX**: Order of 'time' and 'value' arguments reversed.
More Detail
-----------
Connection Pools
^^^^^^^^^^^^^^^^
Behind the scenes, redis-py uses a connection pool to manage connections to
a Redis server. By default, each Redis instance you create will in turn create
its own connection pool. You can override this behavior and use an existing
connection pool by passing an already created connection pool instance to the
connection_pool argument of the Redis class. You may choose to do this in order
to implement client side sharding or have finer grain control of how
connections are managed.
.. code-block:: pycon
>>> pool = redis.ConnectionPool(host='localhost', port=6379, db=0)
>>> r = redis.Redis(connection_pool=pool)
Connections
^^^^^^^^^^^
ConnectionPools manage a set of Connection instances. redis-py ships with two
types of Connections. The default, Connection, is a normal TCP socket based
connection. The UnixDomainSocketConnection allows for clients running on the
same device as the server to connect via a unix domain socket. To use a
UnixDomainSocketConnection connection, simply pass the unix_socket_path
argument, which is a string to the unix domain socket file. Additionally, make
sure the unixsocket parameter is defined in your redis.conf file. It's
commented out by default.
.. code-block:: pycon
>>> r = redis.Redis(unix_socket_path='/tmp/redis.sock')
You can create your own Connection subclasses as well. This may be useful if
you want to control the socket behavior within an async framework. To
instantiate a client class using your own connection, you need to create
a connection pool, passing your class to the connection_class argument.
Other keyword parameters your pass to the pool will be passed to the class
specified during initialization.
.. code-block:: pycon
>>> pool = redis.ConnectionPool(connection_class=YourConnectionClass,
your_arg='...', ...)
Parsers
^^^^^^^
Parser classes provide a way to control how responses from the Redis server
are parsed. redis-py ships with two parser classes, the PythonParser and the
HiredisParser. By default, redis-py will attempt to use the HiredisParser if
you have the hiredis module installed and will fallback to the PythonParser
otherwise.
Hiredis is a C library maintained by the core Redis team. Pieter Noordhuis was
kind enough to create Python bindings. Using Hiredis can provide up to a
10x speed improvement in parsing responses from the Redis server. The
performance increase is most noticeable when retrieving many pieces of data,
such as from LRANGE or SMEMBERS operations.
Hiredis is available on PyPI, and can be installed via pip or easy_install
just like redis-py.
.. code-block:: bash
$ pip install hiredis
or
.. code-block:: bash
$ easy_install hiredis
Response Callbacks
^^^^^^^^^^^^^^^^^^
The client class uses a set of callbacks to cast Redis responses to the
appropriate Python type. There are a number of these callbacks defined on
the Redis client class in a dictionary called RESPONSE_CALLBACKS.
Custom callbacks can be added on a per-instance basis using the
set_response_callback method. This method accepts two arguments: a command
name and the callback. Callbacks added in this manner are only valid on the
instance the callback is added to. If you want to define or override a callback
globally, you should make a subclass of the Redis client and add your callback
to its REDIS_CALLBACKS class dictionary.
Response callbacks take at least one parameter: the response from the Redis
server. Keyword arguments may also be accepted in order to further control
how to interpret the response. These keyword arguments are specified during the
command's call to execute_command. The ZRANGE implementation demonstrates the
use of response callback keyword arguments with its "withscores" argument.
Thread Safety
^^^^^^^^^^^^^
Redis client instances can safely be shared between threads. Internally,
connection instances are only retrieved from the connection pool during
command execution, and returned to the pool directly after. Command execution
never modifies state on the client instance.
However, there is one caveat: the Redis SELECT command. The SELECT command
allows you to switch the database currently in use by the connection. That
database remains selected until another is selected or until the connection is
closed. This creates an issue in that connections could be returned to the pool
that are connected to a different database.
As a result, redis-py does not implement the SELECT command on client
instances. If you use multiple Redis databases within the same application, you
should create a separate client instance (and possibly a separate connection
pool) for each database.
It is not safe to pass PubSub or Pipeline objects between threads.
Pipelines
^^^^^^^^^
Pipelines are a subclass of the base Redis class that provide support for
buffering multiple commands to the server in a single request. They can be used
to dramatically increase the performance of groups of commands by reducing the
number of back-and-forth TCP packets between the client and server.
Pipelines are quite simple to use:
.. code-block:: pycon
>>> r = redis.Redis(...)
>>> r.set('bing', 'baz')
>>> # Use the pipeline() method to create a pipeline instance
>>> pipe = r.pipeline()
>>> # The following SET commands are buffered
>>> pipe.set('foo', 'bar')
>>> pipe.get('bing')
>>> # the EXECUTE call sends all buffered commands to the server, returning
>>> # a list of responses, one for each command.
>>> pipe.execute()
[True, 'baz']
For ease of use, all commands being buffered into the pipeline return the
pipeline object itself. Therefore calls can be chained like:
.. code-block:: pycon
>>> pipe.set('foo', 'bar').sadd('faz', 'baz').incr('auto_number').execute()
[True, True, 6]
In addition, pipelines can also ensure the buffered commands are executed
atomically as a group. This happens by default. If you want to disable the
atomic nature of a pipeline but still want to buffer commands, you can turn
off transactions.
.. code-block:: pycon
>>> pipe = r.pipeline(transaction=False)
A common issue occurs when requiring atomic transactions but needing to
retrieve values in Redis prior for use within the transaction. For instance,
let's assume that the INCR command didn't exist and we need to build an atomic
version of INCR in Python.
The completely naive implementation could GET the value, increment it in
Python, and SET the new value back. However, this is not atomic because
multiple clients could be doing this at the same time, each getting the same
value from GET.
Enter the WATCH command. WATCH provides the ability to monitor one or more keys
prior to starting a transaction. If any of those keys change prior the
execution of that transaction, the entire transaction will be canceled and a
WatchError will be raised. To implement our own client-side INCR command, we
could do something like this:
.. code-block:: pycon
>>> with r.pipeline() as pipe:
... while 1:
... try:
... # put a WATCH on the key that holds our sequence value
... pipe.watch('OUR-SEQUENCE-KEY')
... # after WATCHing, the pipeline is put into immediate execution
... # mode until we tell it to start buffering commands again.
... # this allows us to get the current value of our sequence
... current_value = pipe.get('OUR-SEQUENCE-KEY')
... next_value = int(current_value) + 1
... # now we can put the pipeline back into buffered mode with MULTI
... pipe.multi()
... pipe.set('OUR-SEQUENCE-KEY', next_value)
... # and finally, execute the pipeline (the set command)
... pipe.execute()
... # if a WatchError wasn't raised during execution, everything
... # we just did happened atomically.
... break
... except WatchError:
... # another client must have changed 'OUR-SEQUENCE-KEY' between
... # the time we started WATCHing it and the pipeline's execution.
... # our best bet is to just retry.
... continue
Note that, because the Pipeline must bind to a single connection for the
duration of a WATCH, care must be taken to ensure that the connection is
returned to the connection pool by calling the reset() method. If the
Pipeline is used as a context manager (as in the example above) reset()
will be called automatically. Of course you can do this the manual way by
explicity calling reset():
.. code-block:: pycon
>>> pipe = r.pipeline()
>>> while 1:
... try:
... pipe.watch('OUR-SEQUENCE-KEY')
... ...
... pipe.execute()
... break
... except WatchError:
... continue
... finally:
... pipe.reset()
A convenience method named "transaction" exists for handling all the
boilerplate of handling and retrying watch errors. It takes a callable that
should expect a single parameter, a pipeline object, and any number of keys to
be WATCHed. Our client-side INCR command above can be written like this,
which is much easier to read:
.. code-block:: pycon
>>> def client_side_incr(pipe):
... current_value = pipe.get('OUR-SEQUENCE-KEY')
... next_value = int(current_value) + 1
... pipe.multi()
... pipe.set('OUR-SEQUENCE-KEY', next_value)
>>>
>>> r.transaction(client_side_incr, 'OUR-SEQUENCE-KEY')
[True]
Publish / Subscribe
^^^^^^^^^^^^^^^^^^^
redis-py includes a `PubSub` object that subscribes to channels and listens
for new messages. Creating a `PubSub` object is easy.
.. code-block:: pycon
>>> r = redis.StrictRedis(...)
>>> p = r.pubsub()
Once a `PubSub` instance is created, channels and patterns can be subscribed
to.
.. code-block:: pycon
>>> p.subscribe('my-first-channel', 'my-second-channel', ...)
>>> p.psubscribe('my-*', ...)
The `PubSub` instance is now subscribed to those channels/patterns. The
subscription confirmations can be seen by reading messages from the `PubSub`
instance.
.. code-block:: pycon
>>> p.get_message()
{'pattern': None, 'type': 'subscribe', 'channel': 'my-second-channel', 'data': 1L}
>>> p.get_message()
{'pattern': None, 'type': 'subscribe', 'channel': 'my-first-channel', 'data': 2L}
>>> p.get_message()
{'pattern': None, 'type': 'psubscribe', 'channel': 'my-*', 'data': 3L}
Every message read from a `PubSub` instance will be a dictionary with the
following keys.
* **type**: One of the following: 'subscribe', 'unsubscribe', 'psubscribe',
'punsubscribe', 'message', 'pmessage'
* **channel**: The channel [un]subscribed to or the channel a message was
published to
* **pattern**: The pattern that matched a published message's channel. Will be
`None` in all cases except for 'pmessage' types.
* **data**: The message data. With [un]subscribe messages, this value will be
the number of channels and patterns the connection is currently subscribed
to. With [p]message messages, this value will be the actual published
message.
Let's send a message now.
.. code-block:: pycon
# the publish method returns the number matching channel and pattern
# subscriptions. 'my-first-channel' matches both the 'my-first-channel'
# subscription and the 'my-*' pattern subscription, so this message will
# be delivered to 2 channels/patterns
>>> r.publish('my-first-channel', 'some data')
2
>>> p.get_message()
{'channel': 'my-first-channel', 'data': 'some data', 'pattern': None, 'type': 'message'}
>>> p.get_message()
{'channel': 'my-first-channel', 'data': 'some data', 'pattern': 'my-*', 'type': 'pmessage'}
Unsubscribing works just like subscribing. If no arguments are passed to
[p]unsubscribe, all channels or patterns will be unsubscribed from.
.. code-block:: pycon
>>> p.unsubscribe()
>>> p.punsubscribe('my-*')
>>> p.get_message()
{'channel': 'my-second-channel', 'data': 2L, 'pattern': None, 'type': 'unsubscribe'}
>>> p.get_message()
{'channel': 'my-first-channel', 'data': 1L, 'pattern': None, 'type': 'unsubscribe'}
>>> p.get_message()
{'channel': 'my-*', 'data': 0L, 'pattern': None, 'type': 'punsubscribe'}
redis-py also allows you to register callback functions to handle published
messages. Message handlers take a single argument, the message, which is a
dictionary just like the examples above. To subscribe to a channel or pattern
with a message handler, pass the channel or pattern name as a keyword argument
with its value being the callback function.
When a message is read on a channel or pattern with a message handler, the
message dictionary is created and passed to the message handler. In this case,
a `None` value is returned from get_message() since the message was already
handled.
.. code-block:: pycon
>>> def my_handler(message):
... print 'MY HANDLER: ', message['data']
>>> p.subscribe(**{'my-channel': my_handler})
# read the subscribe confirmation message
>>> p.get_message()
{'pattern': None, 'type': 'subscribe', 'channel': 'my-channel', 'data': 1L}
>>> r.publish('my-channel', 'awesome data')
1
# for the message handler to work, we need tell the instance to read data.
# this can be done in several ways (read more below). we'll just use
# the familiar get_message() function for now
>>> message = p.get_message()
MY HANDLER: awesome data
# note here that the my_handler callback printed the string above.
# `message` is None because the message was handled by our handler.
>>> print message
None
If your application is not interested in the (sometimes noisy)
subscribe/unsubscribe confirmation messages, you can ignore them by passing
`ignore_subscribe_messages=True` to `r.pubsub()`. This will cause all
subscribe/unsubscribe messages to be read, but they won't bubble up to your
application.
.. code-block:: pycon
>>> p = r.pubsub(ignore_subscribe_messages=True)
>>> p.subscribe('my-channel')
>>> p.get_message() # hides the subscribe message and returns None
>>> r.publish('my-channel')
1
>>> p.get_message()
{'channel': 'my-channel', data': 'my data', 'pattern': None, 'type': 'message'}
There are three different strategies for reading messages.
The examples above have been using `pubsub.get_message()`. Behind the scenes,
`get_message()` uses the system's 'select' module to quickly poll the
connection's socket. If there's data available to be read, `get_message()` will
read it, format the message and return it or pass it to a message handler. If
there's no data to be read, `get_message()` will immediately return None. This
makes it trivial to integrate into an existing event loop inside your
application.
.. code-block:: pycon
>>> while True:
>>> message = p.get_message()
>>> if message:
>>> # do something with the message
>>> time.sleep(0.001) # be nice to the system :)
Older versions of redis-py only read messages with `pubsub.listen()`. listen()
is a generator that blocks until a message is available. If your application
doesn't need to do anything else but receive and act on messages received from
redis, listen() is an easy way to get up an running.
.. code-block:: pycon
>>> for message in p.listen():
... # do something with the message
The third option runs an event loop in a separate thread.
`pubsub.run_in_thread()` creates a new thread and starts the event loop. The
thread object is returned to the caller of `run_in_thread()`. The caller can
use the `thread.stop()` method to shut down the event loop and thread. Behind
the scenes, this is simply a wrapper around `get_message()` that runs in a
separate thread, essentially creating a tiny non-blocking event loop for you.
`run_in_thread()` takes an optional `sleep_time` argument. If specified, the
event loop will call `time.sleep()` with the value in each iteration of the
loop.
Note: Since we're running in a separate thread, there's no way to handle
messages that aren't automatically handled with registered message handlers.
Therefore, redis-py prevents you from calling `run_in_thread()` if you're
subscribed to patterns or channels that don't have message handlers attached.
.. code-block:: pycon
>>> p.subscribe(**{'my-channel': my_handler})
>>> thread = p.run_in_thread(sleep_time=0.001)
# the event loop is now running in the background processing messages
# when it's time to shut it down...
>>> thread.stop()
A PubSub object adheres to the same encoding semantics as the client instance
it was created from. Any channel or pattern that's unicode will be encoded
using the `charset` specified on the client before being sent to Redis. If the
client's `decode_responses` flag is set the False (the default), the
'channel', 'pattern' and 'data' values in message dictionaries will be byte
strings (str on Python 2, bytes on Python 3). If the client's
`decode_responses` is True, then the 'channel', 'pattern' and 'data' values
will be automatically decoded to unicode strings using the client's `charset`.
PubSub objects remember what channels and patterns they are subscribed to. In
the event of a disconnection such as a network error or timeout, the
PubSub object will re-subscribe to all prior channels and patterns when
reconnecting. Messages that were published while the client was disconnected
cannot be delivered. When you're finished with a PubSub object, call its
`.close()` method to shutdown the connection.
.. code-block:: pycon
>>> p = r.pubsub()
>>> ...
>>> p.close()
LUA Scripting
^^^^^^^^^^^^^
redis-py supports the EVAL, EVALSHA, and SCRIPT commands. However, there are
a number of edge cases that make these commands tedious to use in real world
scenarios. Therefore, redis-py exposes a Script object that makes scripting
much easier to use.
To create a Script instance, use the `register_script` function on a client
instance passing the LUA code as the first argument. `register_script` returns
a Script instance that you can use throughout your code.
The following trivial LUA script accepts two parameters: the name of a key and
a multiplier value. The script fetches the value stored in the key, multiplies
it with the multiplier value and returns the result.
.. code-block:: pycon
>>> r = redis.StrictRedis()
>>> lua = """
... local value = redis.call('GET', KEYS[1])
... value = tonumber(value)
... return value * ARGV[1]"""
>>> multiply = r.register_script(lua)
`multiply` is now a Script instance that is invoked by calling it like a
function. Script instances accept the following optional arguments:
* **keys**: A list of key names that the script will access. This becomes the
KEYS list in LUA.
* **args**: A list of argument values. This becomes the ARGV list in LUA.
* **client**: A redis-py Client or Pipeline instance that will invoke the
script. If client isn't specified, the client that intiially
created the Script instance (the one that `register_script` was
invoked from) will be used.
Continuing the example from above:
.. code-block:: pycon
>>> r.set('foo', 2)
>>> multiply(keys=['foo'], args=[5])
10
The value of key 'foo' is set to 2. When multiply is invoked, the 'foo' key is
passed to the script along with the multiplier value of 5. LUA executes the
script and returns the result, 10.
Script instances can be executed using a different client instance, even one
that points to a completely different Redis server.
.. code-block:: pycon
>>> r2 = redis.StrictRedis('redis2.example.com')
>>> r2.set('foo', 3)
>>> multiply(keys=['foo'], args=[5], client=r2)
15
The Script object ensures that the LUA script is loaded into Redis's script
cache. In the event of a NOSCRIPT error, it will load the script and retry
executing it.
Script objects can also be used in pipelines. The pipeline instance should be
passed as the client argument when calling the script. Care is taken to ensure
that the script is registered in Redis's script cache just prior to pipeline
execution.
.. code-block:: pycon
>>> pipe = r.pipeline()
>>> pipe.set('foo', 5)
>>> multiply(keys=['foo'], args=[5], client=pipe)
>>> pipe.execute()
[True, 25]
Sentinel support
^^^^^^^^^^^^^^^^
redis-py can be used together with `Redis Sentinel <http://redis.io/topics/sentinel>`_
to discover Redis nodes. You need to have at least one Sentinel daemon running
in order to use redis-py's Sentinel support.
Connecting redis-py to the Sentinel instance(s) is easy. You can use a
Sentinel connection to discover the master and slaves network addresses:
.. code-block:: pycon
>>> from redis.sentinel import Sentinel
>>> sentinel = Sentinel([('localhost', 26379)], socket_timeout=0.1)
>>> sentinel.discover_master('mymaster')
('127.0.0.1', 6379)
>>> sentinel.discover_slaves('mymaster')
[('127.0.0.1', 6380)]
You can also create Redis client connections from a Sentinel instnace. You can
connect to either the master (for write operations) or a slave (for read-only
operations).
.. code-block:: pycon
>>> master = sentinel.master_for('mymaster', socket_timeout=0.1)
>>> slave = sentinel.slave_for('mymaster', socket_timeout=0.1)
>>> master.set('foo', 'bar')
>>> slave.get('foo')
'bar'
The master and slave objects are normal StrictRedis instances with their
connection pool bound to the Sentinel instance. When a Sentinel backed client
attempts to establish a connection, it first queries the Sentinel servers to
determine an appropriate host to connect to. If no server is found,
a MasterNotFoundError or SlaveNotFoundError is raised. Both exceptions are
subclasses of ConnectionError.
When trying to connect to a slave client, the Sentinel connection pool will
iterate over the list of slaves until it finds one that can be connected to.
If no slaves can be connected to, a connection will be established with the
master.
See `Guidelines for Redis clients with support for Redis Sentinel
<http://redis.io/topics/sentinel-clients>`_ to learn more about Redis Sentinel.
Scan Iterators
^^^^^^^^^^^^^^
The *SCAN commands introduced in Redis 2.8 can be cumbersome to use. While
these commands are fully supported, redis-py also exposes the following methods
that return Python iterators for convenience: `scan_iter`, `hscan_iter`,
`sscan_iter` and `zscan_iter`.
.. code-block:: pycon
>>> for key, value in (('A', '1'), ('B', '2'), ('C', '3')):
... r.set(key, value)
>>> for key in r.scan_iter():
... print key, r.get(key)
A 1
B 2
C 3
Author
^^^^^^
redis-py is developed and maintained by Andy McCurdy (sedrik@gmail.com).
It can be found here: http://github.com/andymccurdy/redis-py
Special thanks to:
* Ludovico Magnocavallo, author of the original Python Redis client, from
which some of the socket code is still used.
* Alexander Solovyov for ideas on the generic response callback system.
* Paul Hubbard for initial packaging support.
Keywords: Redis,key-value store
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2.6
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.2
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4

673
awx/lib/site-packages/redis/README.rst
View File

@ -1,673 +0,0 @@
redis-py
========
The Python interface to the Redis key-value store.
.. image:: https://secure.travis-ci.org/andymccurdy/redis-py.png?branch=master
:target: http://travis-ci.org/andymccurdy/redis-py
Installation
------------
redis-py requires a running Redis server. See `Redis's quickstart
<http://redis.io/topics/quickstart>`_ for installation instructions.
To install redis-py, simply:
.. code-block:: bash
$ sudo pip install redis
or alternatively (you really should be using pip though):
.. code-block:: bash
$ sudo easy_install redis
or from source:
.. code-block:: bash
$ sudo python setup.py install
Getting Started
---------------
.. code-block:: pycon
>>> import redis
>>> r = redis.StrictRedis(host='localhost', port=6379, db=0)
>>> r.set('foo', 'bar')
True
>>> r.get('foo')
'bar'
API Reference
-------------
The `official Redis command documentation <http://redis.io/commands>`_ does a
great job of explaining each command in detail. redis-py exposes two client
classes that implement these commands. The StrictRedis class attempts to adhere
to the official command syntax. There are a few exceptions:
* **SELECT**: Not implemented. See the explanation in the Thread Safety section
below.
* **DEL**: 'del' is a reserved keyword in the Python syntax. Therefore redis-py
uses 'delete' instead.
* **CONFIG GET|SET**: These are implemented separately as config_get or config_set.
* **MULTI/EXEC**: These are implemented as part of the Pipeline class. The
pipeline is wrapped with the MULTI and EXEC statements by default when it
is executed, which can be disabled by specifying transaction=False.
See more about Pipelines below.
* **SUBSCRIBE/LISTEN**: Similar to pipelines, PubSub is implemented as a separate
class as it places the underlying connection in a state where it can't
execute non-pubsub commands. Calling the pubsub method from the Redis client
will return a PubSub instance where you can subscribe to channels and listen
for messages. You can only call PUBLISH from the Redis client (see
`this comment on issue #151
<https://github.com/andymccurdy/redis-py/issues/151#issuecomment-1545015>`_
for details).
* **SCAN/SSCAN/HSCAN/ZSCAN**: The *SCAN commands are implemented as they
exist in the Redis documentation. In addition, each command has an equivilant
iterator method. These are purely for convenience so the user doesn't have
to keep track of the cursor while iterating. Use the
scan_iter/sscan_iter/hscan_iter/zscan_iter methods for this behavior.
In addition to the changes above, the Redis class, a subclass of StrictRedis,
overrides several other commands to provide backwards compatibility with older
versions of redis-py:
* **LREM**: Order of 'num' and 'value' arguments reversed such that 'num' can
provide a default value of zero.
* **ZADD**: Redis specifies the 'score' argument before 'value'. These were swapped
accidentally when being implemented and not discovered until after people
were already using it. The Redis class expects \*args in the form of:
`name1, score1, name2, score2, ...`
* **SETEX**: Order of 'time' and 'value' arguments reversed.
More Detail
-----------
Connection Pools
^^^^^^^^^^^^^^^^
Behind the scenes, redis-py uses a connection pool to manage connections to
a Redis server. By default, each Redis instance you create will in turn create
its own connection pool. You can override this behavior and use an existing
connection pool by passing an already created connection pool instance to the
connection_pool argument of the Redis class. You may choose to do this in order
to implement client side sharding or have finer grain control of how
connections are managed.
.. code-block:: pycon
>>> pool = redis.ConnectionPool(host='localhost', port=6379, db=0)
>>> r = redis.Redis(connection_pool=pool)
Connections
^^^^^^^^^^^
ConnectionPools manage a set of Connection instances. redis-py ships with two
types of Connections. The default, Connection, is a normal TCP socket based
connection. The UnixDomainSocketConnection allows for clients running on the
same device as the server to connect via a unix domain socket. To use a
UnixDomainSocketConnection connection, simply pass the unix_socket_path
argument, which is a string to the unix domain socket file. Additionally, make
sure the unixsocket parameter is defined in your redis.conf file. It's
commented out by default.
.. code-block:: pycon
>>> r = redis.Redis(unix_socket_path='/tmp/redis.sock')
You can create your own Connection subclasses as well. This may be useful if
you want to control the socket behavior within an async framework. To
instantiate a client class using your own connection, you need to create
a connection pool, passing your class to the connection_class argument.
Other keyword parameters your pass to the pool will be passed to the class
specified during initialization.
.. code-block:: pycon
>>> pool = redis.ConnectionPool(connection_class=YourConnectionClass,
your_arg='...', ...)
Parsers
^^^^^^^
Parser classes provide a way to control how responses from the Redis server
are parsed. redis-py ships with two parser classes, the PythonParser and the
HiredisParser. By default, redis-py will attempt to use the HiredisParser if
you have the hiredis module installed and will fallback to the PythonParser
otherwise.
Hiredis is a C library maintained by the core Redis team. Pieter Noordhuis was
kind enough to create Python bindings. Using Hiredis can provide up to a
10x speed improvement in parsing responses from the Redis server. The
performance increase is most noticeable when retrieving many pieces of data,
such as from LRANGE or SMEMBERS operations.
Hiredis is available on PyPI, and can be installed via pip or easy_install
just like redis-py.
.. code-block:: bash
$ pip install hiredis
or
.. code-block:: bash
$ easy_install hiredis
Response Callbacks
^^^^^^^^^^^^^^^^^^
The client class uses a set of callbacks to cast Redis responses to the
appropriate Python type. There are a number of these callbacks defined on
the Redis client class in a dictionary called RESPONSE_CALLBACKS.
Custom callbacks can be added on a per-instance basis using the
set_response_callback method. This method accepts two arguments: a command
name and the callback. Callbacks added in this manner are only valid on the
instance the callback is added to. If you want to define or override a callback
globally, you should make a subclass of the Redis client and add your callback
to its REDIS_CALLBACKS class dictionary.
Response callbacks take at least one parameter: the response from the Redis
server. Keyword arguments may also be accepted in order to further control
how to interpret the response. These keyword arguments are specified during the
command's call to execute_command. The ZRANGE implementation demonstrates the
use of response callback keyword arguments with its "withscores" argument.
Thread Safety
^^^^^^^^^^^^^
Redis client instances can safely be shared between threads. Internally,
connection instances are only retrieved from the connection pool during
command execution, and returned to the pool directly after. Command execution
never modifies state on the client instance.
However, there is one caveat: the Redis SELECT command. The SELECT command
allows you to switch the database currently in use by the connection. That
database remains selected until another is selected or until the connection is
closed. This creates an issue in that connections could be returned to the pool
that are connected to a different database.
As a result, redis-py does not implement the SELECT command on client
instances. If you use multiple Redis databases within the same application, you
should create a separate client instance (and possibly a separate connection
pool) for each database.
It is not safe to pass PubSub or Pipeline objects between threads.
Pipelines
^^^^^^^^^
Pipelines are a subclass of the base Redis class that provide support for
buffering multiple commands to the server in a single request. They can be used
to dramatically increase the performance of groups of commands by reducing the
number of back-and-forth TCP packets between the client and server.
Pipelines are quite simple to use:
.. code-block:: pycon
>>> r = redis.Redis(...)
>>> r.set('bing', 'baz')
>>> # Use the pipeline() method to create a pipeline instance
>>> pipe = r.pipeline()
>>> # The following SET commands are buffered
>>> pipe.set('foo', 'bar')
>>> pipe.get('bing')
>>> # the EXECUTE call sends all buffered commands to the server, returning
>>> # a list of responses, one for each command.
>>> pipe.execute()
[True, 'baz']
For ease of use, all commands being buffered into the pipeline return the
pipeline object itself. Therefore calls can be chained like:
.. code-block:: pycon
>>> pipe.set('foo', 'bar').sadd('faz', 'baz').incr('auto_number').execute()
[True, True, 6]
In addition, pipelines can also ensure the buffered commands are executed
atomically as a group. This happens by default. If you want to disable the
atomic nature of a pipeline but still want to buffer commands, you can turn
off transactions.
.. code-block:: pycon
>>> pipe = r.pipeline(transaction=False)
A common issue occurs when requiring atomic transactions but needing to
retrieve values in Redis prior for use within the transaction. For instance,
let's assume that the INCR command didn't exist and we need to build an atomic
version of INCR in Python.
The completely naive implementation could GET the value, increment it in
Python, and SET the new value back. However, this is not atomic because
multiple clients could be doing this at the same time, each getting the same
value from GET.
Enter the WATCH command. WATCH provides the ability to monitor one or more keys
prior to starting a transaction. If any of those keys change prior the
execution of that transaction, the entire transaction will be canceled and a
WatchError will be raised. To implement our own client-side INCR command, we
could do something like this:
.. code-block:: pycon
>>> with r.pipeline() as pipe:
... while 1:
... try:
... # put a WATCH on the key that holds our sequence value
... pipe.watch('OUR-SEQUENCE-KEY')
... # after WATCHing, the pipeline is put into immediate execution
... # mode until we tell it to start buffering commands again.
... # this allows us to get the current value of our sequence
... current_value = pipe.get('OUR-SEQUENCE-KEY')
... next_value = int(current_value) + 1
... # now we can put the pipeline back into buffered mode with MULTI
... pipe.multi()
... pipe.set('OUR-SEQUENCE-KEY', next_value)
... # and finally, execute the pipeline (the set command)
... pipe.execute()
... # if a WatchError wasn't raised during execution, everything
... # we just did happened atomically.
... break
... except WatchError:
... # another client must have changed 'OUR-SEQUENCE-KEY' between
... # the time we started WATCHing it and the pipeline's execution.
... # our best bet is to just retry.
... continue
Note that, because the Pipeline must bind to a single connection for the
duration of a WATCH, care must be taken to ensure that the connection is
returned to the connection pool by calling the reset() method. If the
Pipeline is used as a context manager (as in the example above) reset()
will be called automatically. Of course you can do this the manual way by
explicity calling reset():
.. code-block:: pycon
>>> pipe = r.pipeline()
>>> while 1:
... try:
... pipe.watch('OUR-SEQUENCE-KEY')
... ...
... pipe.execute()
... break
... except WatchError:
... continue
... finally:
... pipe.reset()
A convenience method named "transaction" exists for handling all the
boilerplate of handling and retrying watch errors. It takes a callable that
should expect a single parameter, a pipeline object, and any number of keys to
be WATCHed. Our client-side INCR command above can be written like this,
which is much easier to read:
.. code-block:: pycon
>>> def client_side_incr(pipe):
... current_value = pipe.get('OUR-SEQUENCE-KEY')
... next_value = int(current_value) + 1
... pipe.multi()
... pipe.set('OUR-SEQUENCE-KEY', next_value)
>>>
>>> r.transaction(client_side_incr, 'OUR-SEQUENCE-KEY')
[True]
Publish / Subscribe
^^^^^^^^^^^^^^^^^^^
redis-py includes a `PubSub` object that subscribes to channels and listens
for new messages. Creating a `PubSub` object is easy.
.. code-block:: pycon
>>> r = redis.StrictRedis(...)
>>> p = r.pubsub()
Once a `PubSub` instance is created, channels and patterns can be subscribed
to.
.. code-block:: pycon
>>> p.subscribe('my-first-channel', 'my-second-channel', ...)
>>> p.psubscribe('my-*', ...)
The `PubSub` instance is now subscribed to those channels/patterns. The
subscription confirmations can be seen by reading messages from the `PubSub`
instance.
.. code-block:: pycon
>>> p.get_message()
{'pattern': None, 'type': 'subscribe', 'channel': 'my-second-channel', 'data': 1L}
>>> p.get_message()
{'pattern': None, 'type': 'subscribe', 'channel': 'my-first-channel', 'data': 2L}
>>> p.get_message()
{'pattern': None, 'type': 'psubscribe', 'channel': 'my-*', 'data': 3L}
Every message read from a `PubSub` instance will be a dictionary with the
following keys.
* **type**: One of the following: 'subscribe', 'unsubscribe', 'psubscribe',
'punsubscribe', 'message', 'pmessage'
* **channel**: The channel [un]subscribed to or the channel a message was
published to
* **pattern**: The pattern that matched a published message's channel. Will be
`None` in all cases except for 'pmessage' types.
* **data**: The message data. With [un]subscribe messages, this value will be
the number of channels and patterns the connection is currently subscribed
to. With [p]message messages, this value will be the actual published
message.
Let's send a message now.
.. code-block:: pycon
# the publish method returns the number matching channel and pattern
# subscriptions. 'my-first-channel' matches both the 'my-first-channel'
# subscription and the 'my-*' pattern subscription, so this message will
# be delivered to 2 channels/patterns
>>> r.publish('my-first-channel', 'some data')
2
>>> p.get_message()
{'channel': 'my-first-channel', 'data': 'some data', 'pattern': None, 'type': 'message'}
>>> p.get_message()
{'channel': 'my-first-channel', 'data': 'some data', 'pattern': 'my-*', 'type': 'pmessage'}
Unsubscribing works just like subscribing. If no arguments are passed to
[p]unsubscribe, all channels or patterns will be unsubscribed from.
.. code-block:: pycon
>>> p.unsubscribe()
>>> p.punsubscribe('my-*')
>>> p.get_message()
{'channel': 'my-second-channel', 'data': 2L, 'pattern': None, 'type': 'unsubscribe'}
>>> p.get_message()
{'channel': 'my-first-channel', 'data': 1L, 'pattern': None, 'type': 'unsubscribe'}
>>> p.get_message()
{'channel': 'my-*', 'data': 0L, 'pattern': None, 'type': 'punsubscribe'}
redis-py also allows you to register callback functions to handle published
messages. Message handlers take a single argument, the message, which is a
dictionary just like the examples above. To subscribe to a channel or pattern
with a message handler, pass the channel or pattern name as a keyword argument
with its value being the callback function.
When a message is read on a channel or pattern with a message handler, the
message dictionary is created and passed to the message handler. In this case,
a `None` value is returned from get_message() since the message was already
handled.
.. code-block:: pycon
>>> def my_handler(message):
... print 'MY HANDLER: ', message['data']
>>> p.subscribe(**{'my-channel': my_handler})
# read the subscribe confirmation message
>>> p.get_message()
{'pattern': None, 'type': 'subscribe', 'channel': 'my-channel', 'data': 1L}
>>> r.publish('my-channel', 'awesome data')
1
# for the message handler to work, we need tell the instance to read data.
# this can be done in several ways (read more below). we'll just use
# the familiar get_message() function for now
>>> message = p.get_message()
MY HANDLER: awesome data
# note here that the my_handler callback printed the string above.
# `message` is None because the message was handled by our handler.
>>> print message
None
If your application is not interested in the (sometimes noisy)
subscribe/unsubscribe confirmation messages, you can ignore them by passing
`ignore_subscribe_messages=True` to `r.pubsub()`. This will cause all
subscribe/unsubscribe messages to be read, but they won't bubble up to your
application.
.. code-block:: pycon
>>> p = r.pubsub(ignore_subscribe_messages=True)
>>> p.subscribe('my-channel')
>>> p.get_message() # hides the subscribe message and returns None
>>> r.publish('my-channel')
1
>>> p.get_message()
{'channel': 'my-channel', data': 'my data', 'pattern': None, 'type': 'message'}
There are three different strategies for reading messages.
The examples above have been using `pubsub.get_message()`. Behind the scenes,
`get_message()` uses the system's 'select' module to quickly poll the
connection's socket. If there's data available to be read, `get_message()` will
read it, format the message and return it or pass it to a message handler. If
there's no data to be read, `get_message()` will immediately return None. This
makes it trivial to integrate into an existing event loop inside your
application.
.. code-block:: pycon
>>> while True:
>>> message = p.get_message()
>>> if message:
>>> # do something with the message
>>> time.sleep(0.001) # be nice to the system :)
Older versions of redis-py only read messages with `pubsub.listen()`. listen()
is a generator that blocks until a message is available. If your application
doesn't need to do anything else but receive and act on messages received from
redis, listen() is an easy way to get up an running.
.. code-block:: pycon
>>> for message in p.listen():
... # do something with the message
The third option runs an event loop in a separate thread.
`pubsub.run_in_thread()` creates a new thread and starts the event loop. The
thread object is returned to the caller of `run_in_thread()`. The caller can
use the `thread.stop()` method to shut down the event loop and thread. Behind
the scenes, this is simply a wrapper around `get_message()` that runs in a
separate thread, essentially creating a tiny non-blocking event loop for you.
`run_in_thread()` takes an optional `sleep_time` argument. If specified, the
event loop will call `time.sleep()` with the value in each iteration of the
loop.
Note: Since we're running in a separate thread, there's no way to handle
messages that aren't automatically handled with registered message handlers.
Therefore, redis-py prevents you from calling `run_in_thread()` if you're
subscribed to patterns or channels that don't have message handlers attached.
.. code-block:: pycon
>>> p.subscribe(**{'my-channel': my_handler})
>>> thread = p.run_in_thread(sleep_time=0.001)
# the event loop is now running in the background processing messages
# when it's time to shut it down...
>>> thread.stop()
A PubSub object adheres to the same encoding semantics as the client instance
it was created from. Any channel or pattern that's unicode will be encoded
using the `charset` specified on the client before being sent to Redis. If the
client's `decode_responses` flag is set the False (the default), the
'channel', 'pattern' and 'data' values in message dictionaries will be byte
strings (str on Python 2, bytes on Python 3). If the client's
`decode_responses` is True, then the 'channel', 'pattern' and 'data' values
will be automatically decoded to unicode strings using the client's `charset`.
PubSub objects remember what channels and patterns they are subscribed to. In
the event of a disconnection such as a network error or timeout, the
PubSub object will re-subscribe to all prior channels and patterns when
reconnecting. Messages that were published while the client was disconnected
cannot be delivered. When you're finished with a PubSub object, call its
`.close()` method to shutdown the connection.
.. code-block:: pycon
>>> p = r.pubsub()
>>> ...
>>> p.close()
LUA Scripting
^^^^^^^^^^^^^
redis-py supports the EVAL, EVALSHA, and SCRIPT commands. However, there are
a number of edge cases that make these commands tedious to use in real world
scenarios. Therefore, redis-py exposes a Script object that makes scripting
much easier to use.
To create a Script instance, use the `register_script` function on a client
instance passing the LUA code as the first argument. `register_script` returns
a Script instance that you can use throughout your code.
The following trivial LUA script accepts two parameters: the name of a key and
a multiplier value. The script fetches the value stored in the key, multiplies
it with the multiplier value and returns the result.
.. code-block:: pycon
>>> r = redis.StrictRedis()
>>> lua = """
... local value = redis.call('GET', KEYS[1])
... value = tonumber(value)
... return value * ARGV[1]"""
>>> multiply = r.register_script(lua)
`multiply` is now a Script instance that is invoked by calling it like a
function. Script instances accept the following optional arguments:
* **keys**: A list of key names that the script will access. This becomes the
KEYS list in LUA.
* **args**: A list of argument values. This becomes the ARGV list in LUA.
* **client**: A redis-py Client or Pipeline instance that will invoke the
script. If client isn't specified, the client that intiially
created the Script instance (the one that `register_script` was
invoked from) will be used.
Continuing the example from above:
.. code-block:: pycon
>>> r.set('foo', 2)
>>> multiply(keys=['foo'], args=[5])
10
The value of key 'foo' is set to 2. When multiply is invoked, the 'foo' key is
passed to the script along with the multiplier value of 5. LUA executes the
script and returns the result, 10.
Script instances can be executed using a different client instance, even one
that points to a completely different Redis server.
.. code-block:: pycon
>>> r2 = redis.StrictRedis('redis2.example.com')
>>> r2.set('foo', 3)
>>> multiply(keys=['foo'], args=[5], client=r2)
15
The Script object ensures that the LUA script is loaded into Redis's script
cache. In the event of a NOSCRIPT error, it will load the script and retry
executing it.
Script objects can also be used in pipelines. The pipeline instance should be
passed as the client argument when calling the script. Care is taken to ensure
that the script is registered in Redis's script cache just prior to pipeline
execution.
.. code-block:: pycon
>>> pipe = r.pipeline()
>>> pipe.set('foo', 5)
>>> multiply(keys=['foo'], args=[5], client=pipe)
>>> pipe.execute()
[True, 25]
Sentinel support
^^^^^^^^^^^^^^^^
redis-py can be used together with `Redis Sentinel <http://redis.io/topics/sentinel>`_
to discover Redis nodes. You need to have at least one Sentinel daemon running
in order to use redis-py's Sentinel support.
Connecting redis-py to the Sentinel instance(s) is easy. You can use a
Sentinel connection to discover the master and slaves network addresses:
.. code-block:: pycon
>>> from redis.sentinel import Sentinel
>>> sentinel = Sentinel([('localhost', 26379)], socket_timeout=0.1)
>>> sentinel.discover_master('mymaster')
('127.0.0.1', 6379)
>>> sentinel.discover_slaves('mymaster')
[('127.0.0.1', 6380)]
You can also create Redis client connections from a Sentinel instnace. You can
connect to either the master (for write operations) or a slave (for read-only
operations).
.. code-block:: pycon
>>> master = sentinel.master_for('mymaster', socket_timeout=0.1)
>>> slave = sentinel.slave_for('mymaster', socket_timeout=0.1)
>>> master.set('foo', 'bar')
>>> slave.get('foo')
'bar'
The master and slave objects are normal StrictRedis instances with their
connection pool bound to the Sentinel instance. When a Sentinel backed client
attempts to establish a connection, it first queries the Sentinel servers to
determine an appropriate host to connect to. If no server is found,
a MasterNotFoundError or SlaveNotFoundError is raised. Both exceptions are
subclasses of ConnectionError.
When trying to connect to a slave client, the Sentinel connection pool will
iterate over the list of slaves until it finds one that can be connected to.
If no slaves can be connected to, a connection will be established with the
master.
See `Guidelines for Redis clients with support for Redis Sentinel
<http://redis.io/topics/sentinel-clients>`_ to learn more about Redis Sentinel.
Scan Iterators
^^^^^^^^^^^^^^
The *SCAN commands introduced in Redis 2.8 can be cumbersome to use. While
these commands are fully supported, redis-py also exposes the following methods
that return Python iterators for convenience: `scan_iter`, `hscan_iter`,
`sscan_iter` and `zscan_iter`.
.. code-block:: pycon
>>> for key, value in (('A', '1'), ('B', '2'), ('C', '3')):
... r.set(key, value)
>>> for key in r.scan_iter():
... print key, r.get(key)
A 1
B 2
C 3
Author
^^^^^^
redis-py is developed and maintained by Andy McCurdy (sedrik@gmail.com).
It can be found here: http://github.com/andymccurdy/redis-py
Special thanks to:
* Ludovico Magnocavallo, author of the original Python Redis client, from
which some of the socket code is still used.
* Alexander Solovyov for ideas on the generic response callback system.
* Paul Hubbard for initial packaging support.

0
awx/lib/site-packages/redis/redis/__init__.py → awx/lib/site-packages/redis/__init__.py
View File

0
awx/lib/site-packages/redis/redis/_compat.py → awx/lib/site-packages/redis/_compat.py
View File

0
awx/lib/site-packages/redis/redis/client.py → awx/lib/site-packages/redis/client.py
View File

0
awx/lib/site-packages/redis/redis/connection.py → awx/lib/site-packages/redis/connection.py
View File

0
awx/lib/site-packages/redis/redis/exceptions.py → awx/lib/site-packages/redis/exceptions.py
View File

0
awx/lib/site-packages/redis/redis/lock.py → awx/lib/site-packages/redis/lock.py
View File

0
awx/lib/site-packages/redis/redis/sentinel.py → awx/lib/site-packages/redis/sentinel.py
View File

5
awx/lib/site-packages/redis/setup.cfg
View File

@ -1,5 +0,0 @@
[egg_info]
tag_build =
tag_date = 0
tag_svn_revision = 0

61
awx/lib/site-packages/redis/setup.py
View File

@ -1,61 +0,0 @@
#!/usr/bin/env python
import os
import sys
from redis import __version__
try:
from setuptools import setup
from setuptools.command.test import test as TestCommand
class PyTest(TestCommand):
def finalize_options(self):
TestCommand.finalize_options(self)
self.test_args = []
self.test_suite = True
def run_tests(self):
# import here, because outside the eggs aren't loaded
import pytest
errno = pytest.main(self.test_args)
sys.exit(errno)
except ImportError:
from distutils.core import setup
PyTest = lambda x: x
f = open(os.path.join(os.path.dirname(__file__), 'README.rst'))
long_description = f.read()
f.close()
setup(
name='redis',
version=__version__,
description='Python client for Redis key-value store',
long_description=long_description,
url='http://github.com/andymccurdy/redis-py',
author='Andy McCurdy',
author_email='sedrik@gmail.com',
maintainer='Andy McCurdy',
maintainer_email='sedrik@gmail.com',
keywords=['Redis', 'key-value store'],
license='MIT',
packages=['redis'],
tests_require=['pytest>=2.5.0'],
cmdclass={'test': PyTest},
classifiers=[
'Development Status :: 5 - Production/Stable',
'Environment :: Console',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 2.6',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.2',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
]
)

0
awx/lib/site-packages/redis/tests/__init__.py
View File

46
awx/lib/site-packages/redis/tests/conftest.py
View File

@ -1,46 +0,0 @@
import pytest
import redis
from distutils.version import StrictVersion
_REDIS_VERSIONS = {}
def get_version(**kwargs):
params = {'host': 'localhost', 'port': 6379, 'db': 9}
params.update(kwargs)
key = '%s:%s' % (params['host'], params['port'])
if key not in _REDIS_VERSIONS:
client = redis.Redis(**params)
_REDIS_VERSIONS[key] = client.info()['redis_version']
client.connection_pool.disconnect()
return _REDIS_VERSIONS[key]
def _get_client(cls, request=None, **kwargs):
params = {'host': 'localhost', 'port': 6379, 'db': 9}
params.update(kwargs)
client = cls(**params)
client.flushdb()
if request:
def teardown():
client.flushdb()
client.connection_pool.disconnect()
request.addfinalizer(teardown)
return client
def skip_if_server_version_lt(min_version):
check = StrictVersion(get_version()) < StrictVersion(min_version)
return pytest.mark.skipif(check, reason="")
@pytest.fixture()
def r(request, **kwargs):
return _get_client(redis.Redis, request, **kwargs)
@pytest.fixture()
def sr(request, **kwargs):
return _get_client(redis.StrictRedis, request, **kwargs)

1441
awx/lib/site-packages/redis/tests/test_commands.py
View File

File diff suppressed because it is too large Load Diff

402
awx/lib/site-packages/redis/tests/test_connection_pool.py
View File

@ -1,402 +0,0 @@
from __future__ import with_statement
import os
import pytest
import redis
import time
import re
from threading import Thread
from redis.connection import ssl_available
from .conftest import skip_if_server_version_lt
class DummyConnection(object):
description_format = "DummyConnection<>"
def __init__(self, **kwargs):
self.kwargs = kwargs
self.pid = os.getpid()
class TestConnectionPool(object):
def get_pool(self, connection_kwargs=None, max_connections=None,
connection_class=DummyConnection):
connection_kwargs = connection_kwargs or {}
pool = redis.ConnectionPool(
connection_class=connection_class,
max_connections=max_connections,
**connection_kwargs)
return pool
def test_connection_creation(self):
connection_kwargs = {'foo': 'bar', 'biz': 'baz'}
pool = self.get_pool(connection_kwargs=connection_kwargs)
connection = pool.get_connection('_')
assert isinstance(connection, DummyConnection)
assert connection.kwargs == connection_kwargs
def test_multiple_connections(self):
pool = self.get_pool()
c1 = pool.get_connection('_')
c2 = pool.get_connection('_')
assert c1 != c2
def test_max_connections(self):
pool = self.get_pool(max_connections=2)
pool.get_connection('_')
pool.get_connection('_')
with pytest.raises(redis.ConnectionError):
pool.get_connection('_')
def test_reuse_previously_released_connection(self):
pool = self.get_pool()
c1 = pool.get_connection('_')
pool.release(c1)
c2 = pool.get_connection('_')
assert c1 == c2
def test_repr_contains_db_info_tcp(self):
connection_kwargs = {'host': 'localhost', 'port': 6379, 'db': 1}
pool = self.get_pool(connection_kwargs=connection_kwargs,
connection_class=redis.Connection)
expected = 'ConnectionPool<Connection<host=localhost,port=6379,db=1>>'
assert repr(pool) == expected
def test_repr_contains_db_info_unix(self):
connection_kwargs = {'path': '/abc', 'db': 1}
pool = self.get_pool(connection_kwargs=connection_kwargs,
connection_class=redis.UnixDomainSocketConnection)
expected = 'ConnectionPool<UnixDomainSocketConnection<path=/abc,db=1>>'
assert repr(pool) == expected
class TestBlockingConnectionPool(object):
def get_pool(self, connection_kwargs=None, max_connections=10, timeout=20):
connection_kwargs = connection_kwargs or {}
pool = redis.BlockingConnectionPool(connection_class=DummyConnection,
max_connections=max_connections,
timeout=timeout,
**connection_kwargs)
return pool
def test_connection_creation(self):
connection_kwargs = {'foo': 'bar', 'biz': 'baz'}
pool = self.get_pool(connection_kwargs=connection_kwargs)
connection = pool.get_connection('_')
assert isinstance(connection, DummyConnection)
assert connection.kwargs == connection_kwargs
def test_multiple_connections(self):
pool = self.get_pool()
c1 = pool.get_connection('_')
c2 = pool.get_connection('_')
assert c1 != c2
def test_connection_pool_blocks_until_timeout(self):
"When out of connections, block for timeout seconds, then raise"
pool = self.get_pool(max_connections=1, timeout=0.1)
pool.get_connection('_')
start = time.time()
with pytest.raises(redis.ConnectionError):
pool.get_connection('_')
# we should have waited at least 0.1 seconds
assert time.time() - start >= 0.1
def connection_pool_blocks_until_another_connection_released(self):
"""
When out of connections, block until another connection is released
to the pool
"""
pool = self.get_pool(max_connections=1, timeout=2)
c1 = pool.get_connection('_')
def target():
time.sleep(0.1)
pool.release(c1)
Thread(target=target).start()
start = time.time()
pool.get_connection('_')
assert time.time() - start >= 0.1
def test_reuse_previously_released_connection(self):
pool = self.get_pool()
c1 = pool.get_connection('_')
pool.release(c1)
c2 = pool.get_connection('_')
assert c1 == c2
def test_repr_contains_db_info_tcp(self):
pool = redis.ConnectionPool(host='localhost', port=6379, db=0)
expected = 'ConnectionPool<Connection<host=localhost,port=6379,db=0>>'
assert repr(pool) == expected
def test_repr_contains_db_info_unix(self):
pool = redis.ConnectionPool(
connection_class=redis.UnixDomainSocketConnection,
path='abc',
db=0,
)
expected = 'ConnectionPool<UnixDomainSocketConnection<path=abc,db=0>>'
assert repr(pool) == expected
class TestConnectionPoolURLParsing(object):
def test_defaults(self):
pool = redis.ConnectionPool.from_url('redis://localhost')
assert pool.connection_class == redis.Connection
assert pool.connection_kwargs == {
'host': 'localhost',
'port': 6379,
'db': 0,
'password': None,
}
def test_hostname(self):
pool = redis.ConnectionPool.from_url('redis://myhost')
assert pool.connection_class == redis.Connection
assert pool.connection_kwargs == {
'host': 'myhost',
'port': 6379,
'db': 0,
'password': None,
}
def test_port(self):
pool = redis.ConnectionPool.from_url('redis://localhost:6380')
assert pool.connection_class == redis.Connection
assert pool.connection_kwargs == {
'host': 'localhost',
'port': 6380,
'db': 0,
'password': None,
}
def test_password(self):
pool = redis.ConnectionPool.from_url('redis://:mypassword@localhost')
assert pool.connection_class == redis.Connection
assert pool.connection_kwargs == {
'host': 'localhost',
'port': 6379,
'db': 0,
'password': 'mypassword',
}
def test_db_as_argument(self):
pool = redis.ConnectionPool.from_url('redis://localhost', db='1')
assert pool.connection_class == redis.Connection
assert pool.connection_kwargs == {
'host': 'localhost',
'port': 6379,
'db': 1,
'password': None,
}
def test_db_in_path(self):
pool = redis.ConnectionPool.from_url('redis://localhost/2', db='1')
assert pool.connection_class == redis.Connection
assert pool.connection_kwargs == {
'host': 'localhost',
'port': 6379,
'db': 2,
'password': None,
}
def test_db_in_querystring(self):
pool = redis.ConnectionPool.from_url('redis://localhost/2?db=3',
db='1')
assert pool.connection_class == redis.Connection
assert pool.connection_kwargs == {
'host': 'localhost',
'port': 6379,
'db': 3,
'password': None,
}
def test_extra_querystring_options(self):
pool = redis.ConnectionPool.from_url('redis://localhost?a=1&b=2')
assert pool.connection_class == redis.Connection
assert pool.connection_kwargs == {
'host': 'localhost',
'port': 6379,
'db': 0,
'password': None,
'a': '1',
'b': '2'
}
def test_calling_from_subclass_returns_correct_instance(self):
pool = redis.BlockingConnectionPool.from_url('redis://localhost')
assert isinstance(pool, redis.BlockingConnectionPool)
def test_client_creates_connection_pool(self):
r = redis.StrictRedis.from_url('redis://myhost')
assert r.connection_pool.connection_class == redis.Connection
assert r.connection_pool.connection_kwargs == {
'host': 'myhost',
'port': 6379,
'db': 0,
'password': None,
}
class TestConnectionPoolUnixSocketURLParsing(object):
def test_defaults(self):
pool = redis.ConnectionPool.from_url('unix:///socket')
assert pool.connection_class == redis.UnixDomainSocketConnection
assert pool.connection_kwargs == {
'path': '/socket',
'db': 0,
'password': None,
}
def test_password(self):
pool = redis.ConnectionPool.from_url('unix://:mypassword@/socket')
assert pool.connection_class == redis.UnixDomainSocketConnection
assert pool.connection_kwargs == {
'path': '/socket',
'db': 0,
'password': 'mypassword',
}
def test_db_as_argument(self):
pool = redis.ConnectionPool.from_url('unix:///socket', db=1)
assert pool.connection_class == redis.UnixDomainSocketConnection
assert pool.connection_kwargs == {
'path': '/socket',
'db': 1,
'password': None,
}
def test_db_in_querystring(self):
pool = redis.ConnectionPool.from_url('unix:///socket?db=2', db=1)
assert pool.connection_class == redis.UnixDomainSocketConnection
assert pool.connection_kwargs == {
'path': '/socket',
'db': 2,
'password': None,
}
def test_extra_querystring_options(self):
pool = redis.ConnectionPool.from_url('unix:///socket?a=1&b=2')
assert pool.connection_class == redis.UnixDomainSocketConnection
assert pool.connection_kwargs == {
'path': '/socket',
'db': 0,
'password': None,
'a': '1',
'b': '2'
}
class TestSSLConnectionURLParsing(object):
@pytest.mark.skipif(not ssl_available, reason="SSL not installed")
def test_defaults(self):
pool = redis.ConnectionPool.from_url('rediss://localhost')
assert pool.connection_class == redis.SSLConnection
assert pool.connection_kwargs == {
'host': 'localhost',
'port': 6379,
'db': 0,
'password': None,
}
@pytest.mark.skipif(not ssl_available, reason="SSL not installed")
def test_cert_reqs_options(self):
import ssl
pool = redis.ConnectionPool.from_url('rediss://?ssl_cert_reqs=none')
assert pool.get_connection('_').cert_reqs == ssl.CERT_NONE
pool = redis.ConnectionPool.from_url(
'rediss://?ssl_cert_reqs=optional')
assert pool.get_connection('_').cert_reqs == ssl.CERT_OPTIONAL
pool = redis.ConnectionPool.from_url(
'rediss://?ssl_cert_reqs=required')
assert pool.get_connection('_').cert_reqs == ssl.CERT_REQUIRED
class TestConnection(object):
def test_on_connect_error(self):
"""
An error in Connection.on_connect should disconnect from the server
see for details: https://github.com/andymccurdy/redis-py/issues/368
"""
# this assumes the Redis server being tested against doesn't have
# 9999 databases ;)
bad_connection = redis.Redis(db=9999)
# an error should be raised on connect
with pytest.raises(redis.RedisError):
bad_connection.info()
pool = bad_connection.connection_pool
assert len(pool._available_connections) == 1
assert not pool._available_connections[0]._sock
@skip_if_server_version_lt('2.8.8')
def test_busy_loading_disconnects_socket(self, r):
"""
If Redis raises a LOADING error, the connection should be
disconnected and a BusyLoadingError raised
"""
with pytest.raises(redis.BusyLoadingError):
r.execute_command('DEBUG', 'ERROR', 'LOADING fake message')
pool = r.connection_pool
assert len(pool._available_connections) == 1
assert not pool._available_connections[0]._sock
@skip_if_server_version_lt('2.8.8')
def test_busy_loading_from_pipeline_immediate_command(self, r):
"""
BusyLoadingErrors should raise from Pipelines that execute a
command immediately, like WATCH does.
"""
pipe = r.pipeline()
with pytest.raises(redis.BusyLoadingError):
pipe.immediate_execute_command('DEBUG', 'ERROR',
'LOADING fake message')
pool = r.connection_pool
assert not pipe.connection
assert len(pool._available_connections) == 1
assert not pool._available_connections[0]._sock
@skip_if_server_version_lt('2.8.8')
def test_busy_loading_from_pipeline(self, r):
"""
BusyLoadingErrors should be raised from a pipeline execution
regardless of the raise_on_error flag.
"""
pipe = r.pipeline()
pipe.execute_command('DEBUG', 'ERROR', 'LOADING fake message')
with pytest.raises(redis.BusyLoadingError):
pipe.execute()
pool = r.connection_pool
assert not pipe.connection
assert len(pool._available_connections) == 1
assert not pool._available_connections[0]._sock
@skip_if_server_version_lt('2.8.8')
def test_read_only_error(self, r):
"READONLY errors get turned in ReadOnlyError exceptions"
with pytest.raises(redis.ReadOnlyError):
r.execute_command('DEBUG', 'ERROR', 'READONLY blah blah')
def test_connect_from_url_tcp(self):
connection = redis.Redis.from_url('redis://localhost')
pool = connection.connection_pool
assert re.match('(.*)<(.*)<(.*)>>', repr(pool)).groups() == (
'ConnectionPool',
'Connection',
'host=localhost,port=6379,db=0',
)
def test_connect_from_url_unix(self):
connection = redis.Redis.from_url('unix:///path/to/socket')
pool = connection.connection_pool
assert re.match('(.*)<(.*)<(.*)>>', repr(pool)).groups() == (
'ConnectionPool',
'UnixDomainSocketConnection',
'path=/path/to/socket,db=0',
)

33
awx/lib/site-packages/redis/tests/test_encoding.py
View File

@ -1,33 +0,0 @@
from __future__ import with_statement
import pytest
from redis._compat import unichr, u, unicode
from .conftest import r as _redis_client
class TestEncoding(object):
@pytest.fixture()
def r(self, request):
return _redis_client(request=request, decode_responses=True)
def test_simple_encoding(self, r):
unicode_string = unichr(3456) + u('abcd') + unichr(3421)
r['unicode-string'] = unicode_string
cached_val = r['unicode-string']
assert isinstance(cached_val, unicode)
assert unicode_string == cached_val
def test_list_encoding(self, r):
unicode_string = unichr(3456) + u('abcd') + unichr(3421)
result = [unicode_string, unicode_string, unicode_string]
r.rpush('a', *result)
assert r.lrange('a', 0, -1) == result
class TestCommandsAndTokensArentEncoded(object):
@pytest.fixture()
def r(self, request):
return _redis_client(request=request, charset='utf-16')
def test_basic_command(self, r):
r.set('hello', 'world')

167
awx/lib/site-packages/redis/tests/test_lock.py
View File

@ -1,167 +0,0 @@
from __future__ import with_statement
import pytest
import time
from redis.exceptions import LockError, ResponseError
from redis.lock import Lock, LuaLock
class TestLock(object):
lock_class = Lock
def get_lock(self, redis, *args, **kwargs):
kwargs['lock_class'] = self.lock_class
return redis.lock(*args, **kwargs)
def test_lock(self, sr):
lock = self.get_lock(sr, 'foo')
assert lock.acquire(blocking=False)
assert sr.get('foo') == lock.local.token
assert sr.ttl('foo') == -1
lock.release()
assert sr.get('foo') is None
def test_competing_locks(self, sr):
lock1 = self.get_lock(sr, 'foo')
lock2 = self.get_lock(sr, 'foo')
assert lock1.acquire(blocking=False)
assert not lock2.acquire(blocking=False)
lock1.release()
assert lock2.acquire(blocking=False)
assert not lock1.acquire(blocking=False)
lock2.release()
def test_timeout(self, sr):
lock = self.get_lock(sr, 'foo', timeout=10)
assert lock.acquire(blocking=False)
assert 8 < sr.ttl('foo') <= 10
lock.release()
def test_float_timeout(self, sr):
lock = self.get_lock(sr, 'foo', timeout=9.5)
assert lock.acquire(blocking=False)
assert 8 < sr.pttl('foo') <= 9500
lock.release()
def test_blocking_timeout(self, sr):
lock1 = self.get_lock(sr, 'foo')
assert lock1.acquire(blocking=False)
lock2 = self.get_lock(sr, 'foo', blocking_timeout=0.2)
start = time.time()
assert not lock2.acquire()
assert (time.time() - start) > 0.2
lock1.release()
def test_context_manager(self, sr):
# blocking_timeout prevents a deadlock if the lock can't be acquired
# for some reason
with self.get_lock(sr, 'foo', blocking_timeout=0.2) as lock:
assert sr.get('foo') == lock.local.token
assert sr.get('foo') is None
def test_high_sleep_raises_error(self, sr):
"If sleep is higher than timeout, it should raise an error"
with pytest.raises(LockError):
self.get_lock(sr, 'foo', timeout=1, sleep=2)
def test_releasing_unlocked_lock_raises_error(self, sr):
lock = self.get_lock(sr, 'foo')
with pytest.raises(LockError):
lock.release()
def test_releasing_lock_no_longer_owned_raises_error(self, sr):
lock = self.get_lock(sr, 'foo')
lock.acquire(blocking=False)
# manually change the token
sr.set('foo', 'a')
with pytest.raises(LockError):
lock.release()
# even though we errored, the token is still cleared
assert lock.local.token is None
def test_extend_lock(self, sr):
lock = self.get_lock(sr, 'foo', timeout=10)
assert lock.acquire(blocking=False)
assert 8000 < sr.pttl('foo') <= 10000
assert lock.extend(10)
assert 16000 < sr.pttl('foo') <= 20000
lock.release()
def test_extend_lock_float(self, sr):
lock = self.get_lock(sr, 'foo', timeout=10.0)
assert lock.acquire(blocking=False)
assert 8000 < sr.pttl('foo') <= 10000
assert lock.extend(10.0)
assert 16000 < sr.pttl('foo') <= 20000
lock.release()
def test_extending_unlocked_lock_raises_error(self, sr):
lock = self.get_lock(sr, 'foo', timeout=10)
with pytest.raises(LockError):
lock.extend(10)
def test_extending_lock_with_no_timeout_raises_error(self, sr):
lock = self.get_lock(sr, 'foo')
assert lock.acquire(blocking=False)
with pytest.raises(LockError):
lock.extend(10)
lock.release()
def test_extending_lock_no_longer_owned_raises_error(self, sr):
lock = self.get_lock(sr, 'foo')
assert lock.acquire(blocking=False)
sr.set('foo', 'a')
with pytest.raises(LockError):
lock.extend(10)
class TestLuaLock(TestLock):
lock_class = LuaLock
class TestLockClassSelection(object):
def test_lock_class_argument(self, sr):
lock = sr.lock('foo', lock_class=Lock)
assert type(lock) == Lock
lock = sr.lock('foo', lock_class=LuaLock)
assert type(lock) == LuaLock
def test_cached_lualock_flag(self, sr):
try:
sr._use_lua_lock = True
lock = sr.lock('foo')
assert type(lock) == LuaLock
finally:
sr._use_lua_lock = None
def test_cached_lock_flag(self, sr):
try:
sr._use_lua_lock = False
lock = sr.lock('foo')
assert type(lock) == Lock
finally:
sr._use_lua_lock = None
def test_lua_compatible_server(self, sr, monkeypatch):
@classmethod
def mock_register(cls, redis):
return
monkeypatch.setattr(LuaLock, 'register_scripts', mock_register)
try:
lock = sr.lock('foo')
assert type(lock) == LuaLock
assert sr._use_lua_lock is True
finally:
sr._use_lua_lock = None
def test_lua_unavailable(self, sr, monkeypatch):
@classmethod
def mock_register(cls, redis):
raise ResponseError()
monkeypatch.setattr(LuaLock, 'register_scripts', mock_register)
try:
lock = sr.lock('foo')
assert type(lock) == Lock
assert sr._use_lua_lock is False
finally:
sr._use_lua_lock = None

226
awx/lib/site-packages/redis/tests/test_pipeline.py
View File

@ -1,226 +0,0 @@
from __future__ import with_statement
import pytest
import redis
from redis._compat import b, u, unichr, unicode
class TestPipeline(object):
def test_pipeline(self, r):
with r.pipeline() as pipe:
pipe.set('a', 'a1').get('a').zadd('z', z1=1).zadd('z', z2=4)
pipe.zincrby('z', 'z1').zrange('z', 0, 5, withscores=True)
assert pipe.execute() == \
[
True,
b('a1'),
True,
True,
2.0,
[(b('z1'), 2.0), (b('z2'), 4)],
]
def test_pipeline_length(self, r):
with r.pipeline() as pipe:
# Initially empty.
assert len(pipe) == 0
assert not pipe
# Fill 'er up!
pipe.set('a', 'a1').set('b', 'b1').set('c', 'c1')
assert len(pipe) == 3
assert pipe
# Execute calls reset(), so empty once again.
pipe.execute()
assert len(pipe) == 0
assert not pipe
def test_pipeline_no_transaction(self, r):
with r.pipeline(transaction=False) as pipe:
pipe.set('a', 'a1').set('b', 'b1').set('c', 'c1')
assert pipe.execute() == [True, True, True]
assert r['a'] == b('a1')
assert r['b'] == b('b1')
assert r['c'] == b('c1')
def test_pipeline_no_transaction_watch(self, r):
r['a'] = 0
with r.pipeline(transaction=False) as pipe:
pipe.watch('a')
a = pipe.get('a')
pipe.multi()
pipe.set('a', int(a) + 1)
assert pipe.execute() == [True]
def test_pipeline_no_transaction_watch_failure(self, r):
r['a'] = 0
with r.pipeline(transaction=False) as pipe:
pipe.watch('a')
a = pipe.get('a')
r['a'] = 'bad'
pipe.multi()
pipe.set('a', int(a) + 1)
with pytest.raises(redis.WatchError):
pipe.execute()
assert r['a'] == b('bad')
def test_exec_error_in_response(self, r):
"""
an invalid pipeline command at exec time adds the exception instance
to the list of returned values
"""
r['c'] = 'a'
with r.pipeline() as pipe:
pipe.set('a', 1).set('b', 2).lpush('c', 3).set('d', 4)
result = pipe.execute(raise_on_error=False)
assert result[0]
assert r['a'] == b('1')
assert result[1]
assert r['b'] == b('2')
# we can't lpush to a key that's a string value, so this should
# be a ResponseError exception
assert isinstance(result[2], redis.ResponseError)
assert r['c'] == b('a')
# since this isn't a transaction, the other commands after the
# error are still executed
assert result[3]
assert r['d'] == b('4')
# make sure the pipe was restored to a working state
assert pipe.set('z', 'zzz').execute() == [True]
assert r['z'] == b('zzz')
def test_exec_error_raised(self, r):
r['c'] = 'a'
with r.pipeline() as pipe:
pipe.set('a', 1).set('b', 2).lpush('c', 3).set('d', 4)
with pytest.raises(redis.ResponseError) as ex:
pipe.execute()
assert unicode(ex.value).startswith('Command # 3 (LPUSH c 3) of '
'pipeline caused error: ')
# make sure the pipe was restored to a working state
assert pipe.set('z', 'zzz').execute() == [True]
assert r['z'] == b('zzz')
def test_parse_error_raised(self, r):
with r.pipeline() as pipe:
# the zrem is invalid because we don't pass any keys to it
pipe.set('a', 1).zrem('b').set('b', 2)
with pytest.raises(redis.ResponseError) as ex:
pipe.execute()
assert unicode(ex.value).startswith('Command # 2 (ZREM b) of '
'pipeline caused error: ')
# make sure the pipe was restored to a working state
assert pipe.set('z', 'zzz').execute() == [True]
assert r['z'] == b('zzz')
def test_watch_succeed(self, r):
r['a'] = 1
r['b'] = 2
with r.pipeline() as pipe:
pipe.watch('a', 'b')
assert pipe.watching
a_value = pipe.get('a')
b_value = pipe.get('b')
assert a_value == b('1')
assert b_value == b('2')
pipe.multi()
pipe.set('c', 3)
assert pipe.execute() == [True]
assert not pipe.watching
def test_watch_failure(self, r):
r['a'] = 1
r['b'] = 2
with r.pipeline() as pipe:
pipe.watch('a', 'b')
r['b'] = 3
pipe.multi()
pipe.get('a')
with pytest.raises(redis.WatchError):
pipe.execute()
assert not pipe.watching
def test_unwatch(self, r):
r['a'] = 1
r['b'] = 2
with r.pipeline() as pipe:
pipe.watch('a', 'b')
r['b'] = 3
pipe.unwatch()
assert not pipe.watching
pipe.get('a')
assert pipe.execute() == [b('1')]
def test_transaction_callable(self, r):
r['a'] = 1
r['b'] = 2
has_run = []
def my_transaction(pipe):
a_value = pipe.get('a')
assert a_value in (b('1'), b('2'))
b_value = pipe.get('b')
assert b_value == b('2')
# silly run-once code... incr's "a" so WatchError should be raised
# forcing this all to run again. this should incr "a" once to "2"
if not has_run:
r.incr('a')
has_run.append('it has')
pipe.multi()
pipe.set('c', int(a_value) + int(b_value))
result = r.transaction(my_transaction, 'a', 'b')
assert result == [True]
assert r['c'] == b('4')
def test_exec_error_in_no_transaction_pipeline(self, r):
r['a'] = 1
with r.pipeline(transaction=False) as pipe:
pipe.llen('a')
pipe.expire('a', 100)
with pytest.raises(redis.ResponseError) as ex:
pipe.execute()
assert unicode(ex.value).startswith('Command # 1 (LLEN a) of '
'pipeline caused error: ')
assert r['a'] == b('1')
def test_exec_error_in_no_transaction_pipeline_unicode_command(self, r):
key = unichr(3456) + u('abcd') + unichr(3421)
r[key] = 1
with r.pipeline(transaction=False) as pipe:
pipe.llen(key)
pipe.expire(key, 100)
with pytest.raises(redis.ResponseError) as ex:
pipe.execute()
expected = unicode('Command # 1 (LLEN %s) of pipeline caused '
'error: ') % key
assert unicode(ex.value).startswith(expected)
assert r[key] == b('1')

392
awx/lib/site-packages/redis/tests/test_pubsub.py
View File

@ -1,392 +0,0 @@
from __future__ import with_statement
import pytest
import time
import redis
from redis.exceptions import ConnectionError
from redis._compat import basestring, u, unichr
from .conftest import r as _redis_client
def wait_for_message(pubsub, timeout=0.1, ignore_subscribe_messages=False):
now = time.time()
timeout = now + timeout
while now < timeout:
message = pubsub.get_message(
ignore_subscribe_messages=ignore_subscribe_messages)
if message is not None:
return message
time.sleep(0.01)
now = time.time()
return None
def make_message(type, channel, data, pattern=None):
return {
'type': type,
'pattern': pattern and pattern.encode('utf-8') or None,
'channel': channel.encode('utf-8'),
'data': data.encode('utf-8') if isinstance(data, basestring) else data
}
def make_subscribe_test_data(pubsub, type):
if type == 'channel':
return {
'p': pubsub,
'sub_type': 'subscribe',
'unsub_type': 'unsubscribe',
'sub_func': pubsub.subscribe,
'unsub_func': pubsub.unsubscribe,
'keys': ['foo', 'bar', u('uni') + unichr(4456) + u('code')]
}
elif type == 'pattern':
return {
'p': pubsub,
'sub_type': 'psubscribe',
'unsub_type': 'punsubscribe',
'sub_func': pubsub.psubscribe,
'unsub_func': pubsub.punsubscribe,
'keys': ['f*', 'b*', u('uni') + unichr(4456) + u('*')]
}
assert False, 'invalid subscribe type: %s' % type
class TestPubSubSubscribeUnsubscribe(object):
def _test_subscribe_unsubscribe(self, p, sub_type, unsub_type, sub_func,
unsub_func, keys):
for key in keys:
assert sub_func(key) is None
# should be a message for each channel/pattern we just subscribed to
for i, key in enumerate(keys):
assert wait_for_message(p) == make_message(sub_type, key, i + 1)
for key in keys:
assert unsub_func(key) is None
# should be a message for each channel/pattern we just unsubscribed
# from
for i, key in enumerate(keys):
i = len(keys) - 1 - i
assert wait_for_message(p) == make_message(unsub_type, key, i)
def test_channel_subscribe_unsubscribe(self, r):
kwargs = make_subscribe_test_data(r.pubsub(), 'channel')
self._test_subscribe_unsubscribe(**kwargs)
def test_pattern_subscribe_unsubscribe(self, r):
kwargs = make_subscribe_test_data(r.pubsub(), 'pattern')
self._test_subscribe_unsubscribe(**kwargs)
def _test_resubscribe_on_reconnection(self, p, sub_type, unsub_type,
sub_func, unsub_func, keys):
for key in keys:
assert sub_func(key) is None
# should be a message for each channel/pattern we just subscribed to
for i, key in enumerate(keys):
assert wait_for_message(p) == make_message(sub_type, key, i + 1)
# manually disconnect
p.connection.disconnect()
# calling get_message again reconnects and resubscribes
# note, we may not re-subscribe to channels in exactly the same order
# so we have to do some extra checks to make sure we got them all
messages = []
for i in range(len(keys)):
messages.append(wait_for_message(p))
unique_channels = set()
assert len(messages) == len(keys)
for i, message in enumerate(messages):
assert message['type'] == sub_type
assert message['data'] == i + 1
assert isinstance(message['channel'], bytes)
channel = message['channel'].decode('utf-8')
unique_channels.add(channel)
assert len(unique_channels) == len(keys)
for channel in unique_channels:
assert channel in keys
def test_resubscribe_to_channels_on_reconnection(self, r):
kwargs = make_subscribe_test_data(r.pubsub(), 'channel')
self._test_resubscribe_on_reconnection(**kwargs)
def test_resubscribe_to_patterns_on_reconnection(self, r):
kwargs = make_subscribe_test_data(r.pubsub(), 'pattern')
self._test_resubscribe_on_reconnection(**kwargs)
def _test_subscribed_property(self, p, sub_type, unsub_type, sub_func,
unsub_func, keys):
assert p.subscribed is False
sub_func(keys[0])
# we're now subscribed even though we haven't processed the
# reply from the server just yet
assert p.subscribed is True
assert wait_for_message(p) == make_message(sub_type, keys[0], 1)
# we're still subscribed
assert p.subscribed is True
# unsubscribe from all channels
unsub_func()
# we're still technically subscribed until we process the
# response messages from the server
assert p.subscribed is True
assert wait_for_message(p) == make_message(unsub_type, keys[0], 0)
# now we're no longer subscribed as no more messages can be delivered
# to any channels we were listening to
assert p.subscribed is False
# subscribing again flips the flag back
sub_func(keys[0])
assert p.subscribed is True
assert wait_for_message(p) == make_message(sub_type, keys[0], 1)
# unsubscribe again
unsub_func()
assert p.subscribed is True
# subscribe to another channel before reading the unsubscribe response
sub_func(keys[1])
assert p.subscribed is True
# read the unsubscribe for key1
assert wait_for_message(p) == make_message(unsub_type, keys[0], 0)
# we're still subscribed to key2, so subscribed should still be True
assert p.subscribed is True
# read the key2 subscribe message
assert wait_for_message(p) == make_message(sub_type, keys[1], 1)
unsub_func()
# haven't read the message yet, so we're still subscribed
assert p.subscribed is True
assert wait_for_message(p) == make_message(unsub_type, keys[1], 0)
# now we're finally unsubscribed
assert p.subscribed is False
def test_subscribe_property_with_channels(self, r):
kwargs = make_subscribe_test_data(r.pubsub(), 'channel')
self._test_subscribed_property(**kwargs)
def test_subscribe_property_with_patterns(self, r):
kwargs = make_subscribe_test_data(r.pubsub(), 'pattern')
self._test_subscribed_property(**kwargs)
def test_ignore_all_subscribe_messages(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
checks = (
(p.subscribe, 'foo'),
(p.unsubscribe, 'foo'),
(p.psubscribe, 'f*'),
(p.punsubscribe, 'f*'),
)
assert p.subscribed is False
for func, channel in checks:
assert func(channel) is None
assert p.subscribed is True
assert wait_for_message(p) is None
assert p.subscribed is False
def test_ignore_individual_subscribe_messages(self, r):
p = r.pubsub()
checks = (
(p.subscribe, 'foo'),
(p.unsubscribe, 'foo'),
(p.psubscribe, 'f*'),
(p.punsubscribe, 'f*'),
)
assert p.subscribed is False
for func, channel in checks:
assert func(channel) is None
assert p.subscribed is True
message = wait_for_message(p, ignore_subscribe_messages=True)
assert message is None
assert p.subscribed is False
class TestPubSubMessages(object):
def setup_method(self, method):
self.message = None
def message_handler(self, message):
self.message = message
def test_published_message_to_channel(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
p.subscribe('foo')
assert r.publish('foo', 'test message') == 1
message = wait_for_message(p)
assert isinstance(message, dict)
assert message == make_message('message', 'foo', 'test message')
def test_published_message_to_pattern(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
p.subscribe('foo')
p.psubscribe('f*')
# 1 to pattern, 1 to channel
assert r.publish('foo', 'test message') == 2
message1 = wait_for_message(p)
message2 = wait_for_message(p)
assert isinstance(message1, dict)
assert isinstance(message2, dict)
expected = [
make_message('message', 'foo', 'test message'),
make_message('pmessage', 'foo', 'test message', pattern='f*')
]
assert message1 in expected
assert message2 in expected
assert message1 != message2
def test_channel_message_handler(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
p.subscribe(foo=self.message_handler)
assert r.publish('foo', 'test message') == 1
assert wait_for_message(p) is None
assert self.message == make_message('message', 'foo', 'test message')
def test_pattern_message_handler(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
p.psubscribe(**{'f*': self.message_handler})
assert r.publish('foo', 'test message') == 1
assert wait_for_message(p) is None
assert self.message == make_message('pmessage', 'foo', 'test message',
pattern='f*')
def test_unicode_channel_message_handler(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
channel = u('uni') + unichr(4456) + u('code')
channels = {channel: self.message_handler}
p.subscribe(**channels)
assert r.publish(channel, 'test message') == 1
assert wait_for_message(p) is None
assert self.message == make_message('message', channel, 'test message')
def test_unicode_pattern_message_handler(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
pattern = u('uni') + unichr(4456) + u('*')
channel = u('uni') + unichr(4456) + u('code')
p.psubscribe(**{pattern: self.message_handler})
assert r.publish(channel, 'test message') == 1
assert wait_for_message(p) is None
assert self.message == make_message('pmessage', channel,
'test message', pattern=pattern)
class TestPubSubAutoDecoding(object):
"These tests only validate that we get unicode values back"
channel = u('uni') + unichr(4456) + u('code')
pattern = u('uni') + unichr(4456) + u('*')
data = u('abc') + unichr(4458) + u('123')
def make_message(self, type, channel, data, pattern=None):
return {
'type': type,
'channel': channel,
'pattern': pattern,
'data': data
}
def setup_method(self, method):
self.message = None
def message_handler(self, message):
self.message = message
@pytest.fixture()
def r(self, request):
return _redis_client(request=request, decode_responses=True)
def test_channel_subscribe_unsubscribe(self, r):
p = r.pubsub()
p.subscribe(self.channel)
assert wait_for_message(p) == self.make_message('subscribe',
self.channel, 1)
p.unsubscribe(self.channel)
assert wait_for_message(p) == self.make_message('unsubscribe',
self.channel, 0)
def test_pattern_subscribe_unsubscribe(self, r):
p = r.pubsub()
p.psubscribe(self.pattern)
assert wait_for_message(p) == self.make_message('psubscribe',
self.pattern, 1)
p.punsubscribe(self.pattern)
assert wait_for_message(p) == self.make_message('punsubscribe',
self.pattern, 0)
def test_channel_publish(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
p.subscribe(self.channel)
r.publish(self.channel, self.data)
assert wait_for_message(p) == self.make_message('message',
self.channel,
self.data)
def test_pattern_publish(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
p.psubscribe(self.pattern)
r.publish(self.channel, self.data)
assert wait_for_message(p) == self.make_message('pmessage',
self.channel,
self.data,
pattern=self.pattern)
def test_channel_message_handler(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
p.subscribe(**{self.channel: self.message_handler})
r.publish(self.channel, self.data)
assert wait_for_message(p) is None
assert self.message == self.make_message('message', self.channel,
self.data)
# test that we reconnected to the correct channel
p.connection.disconnect()
assert wait_for_message(p) is None # should reconnect
new_data = self.data + u('new data')
r.publish(self.channel, new_data)
assert wait_for_message(p) is None
assert self.message == self.make_message('message', self.channel,
new_data)
def test_pattern_message_handler(self, r):
p = r.pubsub(ignore_subscribe_messages=True)
p.psubscribe(**{self.pattern: self.message_handler})
r.publish(self.channel, self.data)
assert wait_for_message(p) is None
assert self.message == self.make_message('pmessage', self.channel,
self.data,
pattern=self.pattern)
# test that we reconnected to the correct pattern
p.connection.disconnect()
assert wait_for_message(p) is None # should reconnect
new_data = self.data + u('new data')
r.publish(self.channel, new_data)
assert wait_for_message(p) is None
assert self.message == self.make_message('pmessage', self.channel,
new_data,
pattern=self.pattern)
class TestPubSubRedisDown(object):
def test_channel_subscribe(self, r):
r = redis.Redis(host='localhost', port=6390)
p = r.pubsub()
with pytest.raises(ConnectionError):
p.subscribe('foo')

82
awx/lib/site-packages/redis/tests/test_scripting.py
View File

@ -1,82 +0,0 @@
from __future__ import with_statement
import pytest
from redis import exceptions
from redis._compat import b
multiply_script = """
local value = redis.call('GET', KEYS[1])
value = tonumber(value)
return value * ARGV[1]"""
class TestScripting(object):
@pytest.fixture(autouse=True)
def reset_scripts(self, r):
r.script_flush()
def test_eval(self, r):
r.set('a', 2)
# 2 * 3 == 6
assert r.eval(multiply_script, 1, 'a', 3) == 6
def test_evalsha(self, r):
r.set('a', 2)
sha = r.script_load(multiply_script)
# 2 * 3 == 6
assert r.evalsha(sha, 1, 'a', 3) == 6
def test_evalsha_script_not_loaded(self, r):
r.set('a', 2)
sha = r.script_load(multiply_script)
# remove the script from Redis's cache
r.script_flush()
with pytest.raises(exceptions.NoScriptError):
r.evalsha(sha, 1, 'a', 3)
def test_script_loading(self, r):
# get the sha, then clear the cache
sha = r.script_load(multiply_script)
r.script_flush()
assert r.script_exists(sha) == [False]
r.script_load(multiply_script)
assert r.script_exists(sha) == [True]
def test_script_object(self, r):
r.set('a', 2)
multiply = r.register_script(multiply_script)
assert not multiply.sha
# test evalsha fail -> script load + retry
assert multiply(keys=['a'], args=[3]) == 6
assert multiply.sha
assert r.script_exists(multiply.sha) == [True]
# test first evalsha
assert multiply(keys=['a'], args=[3]) == 6
def test_script_object_in_pipeline(self, r):
multiply = r.register_script(multiply_script)
assert not multiply.sha
pipe = r.pipeline()
pipe.set('a', 2)
pipe.get('a')
multiply(keys=['a'], args=[3], client=pipe)
# even though the pipeline wasn't executed yet, we made sure the
# script was loaded and got a valid sha
assert multiply.sha
assert r.script_exists(multiply.sha) == [True]
# [SET worked, GET 'a', result of multiple script]
assert pipe.execute() == [True, b('2'), 6]
# purge the script from redis's cache and re-run the pipeline
# the multiply script object knows it's sha, so it shouldn't get
# reloaded until pipe.execute()
r.script_flush()
pipe = r.pipeline()
pipe.set('a', 2)
pipe.get('a')
assert multiply.sha
multiply(keys=['a'], args=[3], client=pipe)
assert r.script_exists(multiply.sha) == [False]
# [SET worked, GET 'a', result of multiple script]
assert pipe.execute() == [True, b('2'), 6]

173
awx/lib/site-packages/redis/tests/test_sentinel.py
View File

@ -1,173 +0,0 @@
from __future__ import with_statement
import pytest
from redis import exceptions
from redis.sentinel import (Sentinel, SentinelConnectionPool,
MasterNotFoundError, SlaveNotFoundError)
from redis._compat import next
import redis.sentinel
class SentinelTestClient(object):
def __init__(self, cluster, id):
self.cluster = cluster
self.id = id
def sentinel_masters(self):
self.cluster.connection_error_if_down(self)
return {self.cluster.service_name: self.cluster.master}
def sentinel_slaves(self, master_name):
self.cluster.connection_error_if_down(self)
if master_name != self.cluster.service_name:
return []
return self.cluster.slaves
class SentinelTestCluster(object):
def __init__(self, service_name='mymaster', ip='127.0.0.1', port=6379):
self.clients = {}
self.master = {
'ip': ip,
'port': port,
'is_master': True,
'is_sdown': False,
'is_odown': False,
'num-other-sentinels': 0,
}
self.service_name = service_name
self.slaves = []
self.nodes_down = set()
def connection_error_if_down(self, node):
if node.id in self.nodes_down:
raise exceptions.ConnectionError
def client(self, host, port, **kwargs):
return SentinelTestClient(self, (host, port))
@pytest.fixture()
def cluster(request):
def teardown():
redis.sentinel.StrictRedis = saved_StrictRedis
cluster = SentinelTestCluster()
saved_StrictRedis = redis.sentinel.StrictRedis
redis.sentinel.StrictRedis = cluster.client
request.addfinalizer(teardown)
return cluster
@pytest.fixture()
def sentinel(request, cluster):
return Sentinel([('foo', 26379), ('bar', 26379)])
def test_discover_master(sentinel):
address = sentinel.discover_master('mymaster')
assert address == ('127.0.0.1', 6379)
def test_discover_master_error(sentinel):
with pytest.raises(MasterNotFoundError):
sentinel.discover_master('xxx')
def test_discover_master_sentinel_down(cluster, sentinel):
# Put first sentinel 'foo' down
cluster.nodes_down.add(('foo', 26379))
address = sentinel.discover_master('mymaster')
assert address == ('127.0.0.1', 6379)
# 'bar' is now first sentinel
assert sentinel.sentinels[0].id == ('bar', 26379)
def test_master_min_other_sentinels(cluster):
sentinel = Sentinel([('foo', 26379)], min_other_sentinels=1)
# min_other_sentinels
with pytest.raises(MasterNotFoundError):
sentinel.discover_master('mymaster')
cluster.master['num-other-sentinels'] = 2
address = sentinel.discover_master('mymaster')
assert address == ('127.0.0.1', 6379)
def test_master_odown(cluster, sentinel):
cluster.master['is_odown'] = True
with pytest.raises(MasterNotFoundError):
sentinel.discover_master('mymaster')
def test_master_sdown(cluster, sentinel):
cluster.master['is_sdown'] = True
with pytest.raises(MasterNotFoundError):
sentinel.discover_master('mymaster')
def test_discover_slaves(cluster, sentinel):
assert sentinel.discover_slaves('mymaster') == []
cluster.slaves = [
{'ip': 'slave0', 'port': 1234, 'is_odown': False, 'is_sdown': False},
{'ip': 'slave1', 'port': 1234, 'is_odown': False, 'is_sdown': False},
]
assert sentinel.discover_slaves('mymaster') == [
('slave0', 1234), ('slave1', 1234)]
# slave0 -> ODOWN
cluster.slaves[0]['is_odown'] = True
assert sentinel.discover_slaves('mymaster') == [
('slave1', 1234)]
# slave1 -> SDOWN
cluster.slaves[1]['is_sdown'] = True
assert sentinel.discover_slaves('mymaster') == []
cluster.slaves[0]['is_odown'] = False
cluster.slaves[1]['is_sdown'] = False
# node0 -> DOWN
cluster.nodes_down.add(('foo', 26379))
assert sentinel.discover_slaves('mymaster') == [
('slave0', 1234), ('slave1', 1234)]
def test_master_for(cluster, sentinel):
master = sentinel.master_for('mymaster', db=9)
assert master.ping()
assert master.connection_pool.master_address == ('127.0.0.1', 6379)
# Use internal connection check
master = sentinel.master_for('mymaster', db=9, check_connection=True)
assert master.ping()
def test_slave_for(cluster, sentinel):
cluster.slaves = [
{'ip': '127.0.0.1', 'port': 6379,
'is_odown': False, 'is_sdown': False},
]
slave = sentinel.slave_for('mymaster', db=9)
assert slave.ping()
def test_slave_for_slave_not_found_error(cluster, sentinel):
cluster.master['is_odown'] = True
slave = sentinel.slave_for('mymaster', db=9)
with pytest.raises(SlaveNotFoundError):
slave.ping()
def test_slave_round_robin(cluster, sentinel):
cluster.slaves = [
{'ip': 'slave0', 'port': 6379, 'is_odown': False, 'is_sdown': False},
{'ip': 'slave1', 'port': 6379, 'is_odown': False, 'is_sdown': False},
]
pool = SentinelConnectionPool('mymaster', sentinel)
rotator = pool.rotate_slaves()
assert next(rotator) in (('slave0', 6379), ('slave1', 6379))
assert next(rotator) in (('slave0', 6379), ('slave1', 6379))
# Fallback to master
assert next(rotator) == ('127.0.0.1', 6379)
with pytest.raises(SlaveNotFoundError):
next(rotator)

0
awx/lib/site-packages/redis/redis/utils.py → awx/lib/site-packages/redis/utils.py
View File