strutils
- Text manipulation
So much practical programming involves string manipulation, which
Python readily accommodates. Still, there are dozens of basic and
common capabilities missing from the standard library, several of them
provided by strutils
.
- class boltons.strutils.MultiReplace(sub_map, **kwargs)[source]
MultiReplace is a tool for doing multiple find/replace actions in one pass.
Given a mapping of values to be replaced it allows for all of the matching values to be replaced in a single pass which can save a lot of performance on very large strings. In addition to simple replace, it also allows for replacing based on regular expressions.
Keyword Arguments:
- Parameters:
Dictionary Usage:
from boltons import stringutils s = stringutils.MultiReplace({ 'foo': 'zoo', 'cat': 'hat', 'bat': 'kraken' }) new = s.sub('The foo bar cat ate a bat') new == 'The zoo bar hat ate a kraken'
Iterable Usage:
from boltons import stringutils s = stringutils.MultiReplace([ ('foo', 'zoo'), ('cat', 'hat'), ('bat', 'kraken)' ]) new = s.sub('The foo bar cat ate a bat') new == 'The zoo bar hat ate a kraken'
The constructor can be passed a dictionary or other mapping as well as an iterable of tuples. If given an iterable, the substitution will be run in the order the replacement values are specified in the iterable. This is also true if it is given an OrderedDict. If given a dictionary then the order will be non-deterministic:
>>> 'foo bar baz'.replace('foo', 'baz').replace('baz', 'bar') 'bar bar bar' >>> m = MultiReplace({'foo': 'baz', 'baz': 'bar'}) >>> m.sub('foo bar baz') 'baz bar bar'
This is because the order of replacement can matter if you’re inserting something that might be replaced by a later substitution. Pay attention and if you need to rely on order then consider using a list of tuples instead of a dictionary.
- boltons.strutils.a10n(string)[source]
That thing where “internationalization” becomes “i18n”, what’s it called? Abbreviation? Oh wait, no:
a10n
. (It’s actually a form of numeronym.)>>> a10n('abbreviation') 'a10n' >>> a10n('internationalization') 'i18n' >>> a10n('') ''
- boltons.strutils.args2cmd(args, sep=' ')[source]
Return a shell-escaped string version of args, separated by sep, using the same rules as the Microsoft C runtime.
>>> print(args2cmd(['aa', '[bb]', "cc'cc", 'dd"dd'])) aa [bb] cc'cc dd\"dd
As you can see, escaping is through backslashing and not quoting, and double quotes are the only special character. See the comment in the code for more details. Based on internal code from the
subprocess
module.
- boltons.strutils.args2sh(args, sep=' ')[source]
Return a shell-escaped string version of args, separated by sep, based on the rules of sh, bash, and other shells in the Linux/BSD/MacOS ecosystem.
>>> print(args2sh(['aa', '[bb]', "cc'cc", 'dd"dd'])) aa '[bb]' 'cc'"'"'cc' 'dd"dd'
As you can see, arguments with no special characters are not escaped, arguments with special characters are quoted with single quotes, and single quotes themselves are quoted with double quotes. Double quotes are handled like any other special character.
Based on code from the
pipes
/shlex
modules. Also note thatshlex
andargparse
have functions to split and parse strings escaped in this manner.
- boltons.strutils.asciify(text, ignore=False)[source]
Converts a unicode or bytestring, text, into a bytestring with just ascii characters. Performs basic deaccenting for all you Europhiles out there.
Also, a gentle reminder that this is a utility, primarily meant for slugification. Whenever possible, make your application work with unicode, not against it.
- Parameters:
>>> asciify('Beyoncé') == b'Beyonce' True
- boltons.strutils.bytes2human(nbytes, ndigits=0)[source]
Turns an integer value of nbytes into a human readable format. Set ndigits to control how many digits after the decimal point should be shown (default
0
).>>> bytes2human(128991) '126K' >>> bytes2human(100001221) '95M' >>> bytes2human(0, 2) '0.00B'
- boltons.strutils.camel2under(camel_string)[source]
Converts a camelcased string to underscores. Useful for turning a class name into a function name.
>>> camel2under('BasicParseTest') 'basic_parse_test'
- boltons.strutils.cardinalize(unit_noun, count)[source]
Conditionally pluralizes a singular word unit_noun if count is not one, preserving case when possible.
>>> vowels = 'aeiou' >>> print(len(vowels), cardinalize('vowel', len(vowels))) 5 vowels >>> print(3, cardinalize('Wish', 3)) 3 Wishes
- boltons.strutils.complement_int_list(range_string, range_start=0, range_end=None, delim=',', range_delim='-')[source]
Returns range string that is the complement of the one provided as range_string parameter.
These range strings are of the kind produce by
format_int_list()
, and parseable byparse_int_list()
.- Parameters:
range_string (str) – String of comma separated positive integers or ranges (e.g. ‘1,2,4-6,8’). Typical of a custom page range string used in printer dialogs.
range_start (int) – A positive integer from which to start the resulting range. Value is inclusive. Defaults to
0
.range_end (int) – A positive integer from which the produced range is stopped. Value is exclusive. Defaults to the maximum value found in the provided
range_string
.delim (char) – Defaults to ‘,’. Separates integers and contiguous ranges of integers.
range_delim (char) – Defaults to ‘-’. Indicates a contiguous range of integers.
>>> complement_int_list('1,3,5-8,10-11,15') '0,2,4,9,12-14'
>>> complement_int_list('1,3,5-8,10-11,15', range_start=0) '0,2,4,9,12-14'
>>> complement_int_list('1,3,5-8,10-11,15', range_start=1) '2,4,9,12-14'
>>> complement_int_list('1,3,5-8,10-11,15', range_start=2) '2,4,9,12-14'
>>> complement_int_list('1,3,5-8,10-11,15', range_start=3) '4,9,12-14'
>>> complement_int_list('1,3,5-8,10-11,15', range_end=15) '0,2,4,9,12-14'
>>> complement_int_list('1,3,5-8,10-11,15', range_end=14) '0,2,4,9,12-13'
>>> complement_int_list('1,3,5-8,10-11,15', range_end=13) '0,2,4,9,12'
>>> complement_int_list('1,3,5-8,10-11,15', range_end=20) '0,2,4,9,12-14,16-19'
>>> complement_int_list('1,3,5-8,10-11,15', range_end=0) ''
>>> complement_int_list('1,3,5-8,10-11,15', range_start=-1) '0,2,4,9,12-14'
>>> complement_int_list('1,3,5-8,10-11,15', range_end=-1) ''
>>> complement_int_list('1,3,5-8', range_start=1, range_end=1) ''
>>> complement_int_list('1,3,5-8', range_start=2, range_end=2) ''
>>> complement_int_list('1,3,5-8', range_start=2, range_end=3) '2'
>>> complement_int_list('1,3,5-8', range_start=-10, range_end=-5) ''
>>> complement_int_list('1,3,5-8', range_start=20, range_end=10) ''
>>> complement_int_list('') ''
- boltons.strutils.escape_shell_args(args, sep=' ', style=None)[source]
Returns an escaped version of each string in args, according to style.
- Parameters:
args (list) – A list of arguments to escape and join together
sep (str) – The separator used to join the escaped arguments.
style (str) – The style of escaping to use. Can be one of
cmd
orsh
, geared toward Windows and Linux/BSD/etc., respectively. If style isNone
, then it is picked according to the system platform.
See
args2cmd()
andargs2sh()
for details and example output for each style.
- boltons.strutils.find_hashtags(string)[source]
Finds and returns all hashtags in a string, with the hashmark removed. Supports full-width hashmarks for Asian languages and does not false-positive on URL anchors.
>>> find_hashtags('#atag http://asite/#ananchor') ['atag']
find_hashtags
also works with unicode hashtags.
- boltons.strutils.format_int_list(int_list, delim=',', range_delim='-', delim_space=False)[source]
Returns a sorted range string from a list of positive integers (int_list). Contiguous ranges of integers are collapsed to min and max values. Reverse of
parse_int_list()
.- Parameters:
int_list (list) – List of positive integers to be converted into a range string (e.g. [1,2,4,5,6,8]).
delim (char) – Defaults to ‘,’. Separates integers and contiguous ranges of integers.
range_delim (char) – Defaults to ‘-’. Indicates a contiguous range of integers.
delim_space (bool) – Defaults to
False
. IfTrue
, adds a space after all delim characters.
>>> format_int_list([1,3,5,6,7,8,10,11,15]) '1,3,5-8,10-11,15'
- boltons.strutils.gunzip_bytes(bytestring)[source]
The
gzip
module is great if you have a file or file-like object, but what if you just have bytes. StringIO is one possibility, but it’s often faster, easier, and simpler to just use this one-liner. Use this tried-and-true utility function to decompress gzip from bytes.>>> gunzip_bytes(_EMPTY_GZIP_BYTES) == b'' True >>> gunzip_bytes(_NON_EMPTY_GZIP_BYTES).rstrip() == b'bytesahoy!' True
- boltons.strutils.gzip_bytes(bytestring, level=6)[source]
Turn some bytes into some compressed bytes.
>>> len(gzip_bytes(b'a' * 10000)) 46
- Parameters:
Note that all levels of gzip are pretty fast these days, though it’s not really a competitor in compression, at any level.
- boltons.strutils.html2text(html)[source]
Strips tags from HTML text, returning markup-free text. Also, does a best effort replacement of entities like “ ”
>>> r = html2text(u'<a href="#">Test &<em>(Δημώ)</em></a>') >>> r == u'Test &(Δημώ)' True
- boltons.strutils.indent(text, margin, newline='\n', key=<class 'bool'>)[source]
The missing counterpart to the built-in
textwrap.dedent()
.- Parameters:
- boltons.strutils.int_ranges_from_int_list(range_string, delim=',', range_delim='-')[source]
Transform a string of ranges (range_string) into a tuple of tuples.
- Parameters:
range_string (str) – String of comma separated positive integers or ranges (e.g. ‘1,2,4-6,8’). Typical of a custom page range string used in printer dialogs.
delim (char) – Defaults to ‘,’. Separates integers and contiguous ranges of integers.
range_delim (char) – Defaults to ‘-’. Indicates a contiguous range of integers.
>>> int_ranges_from_int_list('1,3,5-8,10-11,15') ((1, 1), (3, 3), (5, 8), (10, 11), (15, 15))
>>> int_ranges_from_int_list('1') ((1, 1),)
>>> int_ranges_from_int_list('') ()
- boltons.strutils.is_ascii(text)[source]
Check if a string or bytestring, text, is composed of ascii characters only. Raises
ValueError
if argument is not text.- Parameters:
text (str) – The string to be checked.
>>> is_ascii('Beyoncé') False >>> is_ascii('Beyonce') True
- boltons.strutils.is_uuid(obj, version=4)[source]
Check the argument is either a valid UUID object or string.
- Parameters:
>>> is_uuid('e682ccca-5a4c-4ef2-9711-73f9ad1e15ea') True >>> is_uuid('0221f0d9-d4b9-11e5-a478-10ddb1c2feb9') False >>> is_uuid('0221f0d9-d4b9-11e5-a478-10ddb1c2feb9', version=1) True
- boltons.strutils.iter_splitlines(text)[source]
Like
str.splitlines()
, but returns an iterator of lines instead of a list. Also similar tofile.next()
, as that also lazily reads and yields lines from a file.This function works with a variety of line endings, but as always, be careful when mixing line endings within a file.
>>> list(iter_splitlines('\nhi\nbye\n')) ['', 'hi', 'bye', ''] >>> list(iter_splitlines('\r\nhi\rbye\r\n')) ['', 'hi', 'bye', ''] >>> list(iter_splitlines('')) []
- boltons.strutils.multi_replace(text, sub_map, **kwargs)[source]
Shortcut function to invoke MultiReplace in a single call.
Example Usage:
from boltons.stringutils import multi_replace new = multi_replace( 'The foo bar cat ate a bat', {'foo': 'zoo', 'cat': 'hat', 'bat': 'kraken'} ) new == 'The zoo bar hat ate a kraken'
- boltons.strutils.ordinalize(number, ext_only=False)[source]
Turns number into its cardinal form, i.e., 1st, 2nd, 3rd, 4th, etc. If the last character isn’t a digit, it returns the string value unchanged.
- Parameters:
>>> print(ordinalize(1)) 1st >>> print(ordinalize(3694839230)) 3694839230th >>> print(ordinalize('hi')) hi >>> print(ordinalize(1515)) 1515th
- boltons.strutils.parse_int_list(range_string, delim=',', range_delim='-')[source]
Returns a sorted list of positive integers based on range_string. Reverse of
format_int_list()
.- Parameters:
range_string (str) – String of comma separated positive integers or ranges (e.g. ‘1,2,4-6,8’). Typical of a custom page range string used in printer dialogs.
delim (char) – Defaults to ‘,’. Separates integers and contiguous ranges of integers.
range_delim (char) – Defaults to ‘-’. Indicates a contiguous range of integers.
>>> parse_int_list('1,3,5-8,10-11,15') [1, 3, 5, 6, 7, 8, 10, 11, 15]
- boltons.strutils.pluralize(word)[source]
Semi-intelligently converts an English word from singular form to plural, preserving case pattern.
>>> pluralize('friend') 'friends' >>> pluralize('enemy') 'enemies' >>> pluralize('Sheep') 'Sheep'
- boltons.strutils.singularize(word)[source]
Semi-intelligently converts an English plural word to its singular form, preserving case pattern.
>>> singularize('chances') 'chance' >>> singularize('Activities') 'Activity' >>> singularize('Glasses') 'Glass' >>> singularize('FEET') 'FOOT'
- boltons.strutils.slugify(text, delim='_', lower=True, ascii=False)[source]
A basic function that turns text full of scary characters (i.e., punctuation and whitespace), into a relatively safe lowercased string separated only by the delimiter specified by delim, which defaults to
_
.The ascii convenience flag will
asciify()
the slug if you require ascii-only slugs.>>> slugify('First post! Hi!!!!~1 ') 'first_post_hi_1'
>>> slugify("Kurt Gödel's pretty cool.", ascii=True) == b'kurt_goedel_s_pretty_cool' True
- boltons.strutils.split_punct_ws(text)[source]
While
str.split()
will split on whitespace,split_punct_ws()
will split on punctuation and whitespace. This used internally byslugify()
, above.>>> split_punct_ws('First post! Hi!!!!~1 ') ['First', 'post', 'Hi', '1']
- boltons.strutils.strip_ansi(text)[source]
Strips ANSI escape codes from text. Useful for the occasional time when a log or redirected output accidentally captures console color codes and the like.
>>> strip_ansi('[0m[1;36mart[46;34m') 'art'
Supports str, bytes and bytearray content as input. Returns the same type as the input.
There’s a lot of ANSI art available for testing on sixteencolors.net. This function does not interpret or render ANSI art, but you can do so with ansi2img or escapes.js.
- boltons.strutils.under2camel(under_string)[source]
Converts an underscored string to camelcased. Useful for turning a function name into a class name.
>>> under2camel('complex_tokenizer') 'ComplexTokenizer'
- boltons.strutils.unit_len(sized_iterable, unit_noun='item')[source]
Returns a plain-English description of an iterable’s
len()
, conditionally pluralized withcardinalize()
, detailed below.>>> print(unit_len(range(10), 'number')) 10 numbers >>> print(unit_len('aeiou', 'vowel')) 5 vowels >>> print(unit_len([], 'worry')) No worries
- boltons.strutils.unwrap_text(text, ending='\n\n')[source]
Unwrap text, the natural complement to
textwrap.wrap()
.>>> text = "Short \n lines \nwrapped\nsmall.\n\nAnother\nparagraph." >>> unwrap_text(text) 'Short lines wrapped small.\n\nAnother paragraph.'
- Parameters:
text – A string to unwrap.
ending (str) – The string to join all unwrapped paragraphs by. Pass
None
to get the list. Defaults to ‘nn’ for compatibility with Markdown and RST.