(This question is related to this one and this one, but those are pre-walking the generator, which is exactly what I want to avoid) I would like to split a generator in chunks. The requirements are: do not pad the chunks: if the number of remaining elements is less than the chunk size, the last chunk must be …
Tag: generator
Tracking how many elements processed in generator
I have a problem in which I process documents from files using python generators. The number of files I need to process are not known in advance. Each file contain records which consumes considerable amount of memory. Due to that, generators are used to process records. Here is the summary of the code I am wo…
Misunderstood python yield
This code below works correct : But this function raises StopIteration. I don’t understand why ? Answer You have: Notice line = f.readline() This only reads 1 line from the file. Compare: with this: yield can only be called once with a particular object or expression. Once it is used by the receiver it …
Merge of lazy streams (using generators) in Python
I’m playing with functional capacities of Python 3 and I tried to implement classical algorithm for calculating Hamming numbers. That’s the numbers which have as prime factors only 2, 3 or 5. First Hamming numbers are 2, 3, 4, 5, 6, 8, 10, 12, 15, 16, 18, 20 and so on. My implementation is the fol…
How to define an empty generator function?
A generator function can be defined by putting the yield keyword in the function’s body: How to define an empty generator function? The following code doesn’t work, since Python cannot know that it is supposed to be a generator function instead of a normal function: I could do something like this: But that wo…
Is enumerate in python lazy?
I’d like to know what happens when I pass the result of a generator function to python’s enumerate(). Example: Is the enumeration iterated lazily, or does it slurp everything into the <enumerate object> first? I’m 99.999% sure it’s lazy, so can I treat it exactly the same as the …
Understanding generators in Python
I am reading the Python cookbook at the moment and am currently looking at generators. I’m finding it hard to get my head round. As I come from a Java background, is there a Java equivalent? The book was speaking about ‘Producer / Consumer’, however when I hear that I think of threading. Wha…
When is not a good time to use python generators?
This is rather the inverse of What can you use Python generator functions for?: python generators, generator expressions, and the itertools module are some of my favorite features of python these days. They’re especially useful when setting up chains of operations to perform on a big pile of data–…