I’m trying to create a new variable in which datetime64[ns] objects are assigned to 5 minute intervals. The new interval variable should span every 5 minute period from 00:00 to 23:55. The criteria for assignment is whether the time of the datetime64[ns] object falls within the corresponding 5 min interval. My actual data has numerous dates in the DateTime variable, but these different dates shouldn’t be taken into account – only the time element matters for assignment.
I’ve simulated this below. This example focuses on the time period circa 23:30 to 23:45, but it should exemplify what I’m trying to achieve for all the intervals from 00:00 to 23:55. I have included two random dates to illustrate how the dates should not have any bearing.
DateTime 2009-02-18 23:32:29 - would map to interval 23:30 2009-02-18 23:34:41 - would map to interval 23:30 2009-02-18 23:35:40 - would map to interval 23.35 2009-02-18 23:39:29 - would map to interval 23:35 2009-02-18 23:39:37 - would map to interval 23:35 2009-02-18 23:40:14 - would map to interval 23:40 2009-02-18 23:43:23 - would map to interval 23:40 2009-02-18 23:44:17 - would map to interval 23:40 ... 2010-03-18 23:31:19 - also maps to interval 23:30 regardless of date 2010-03-18 23:33:31 - also maps to interval 23:30 regardless of date 2010-03-18 23:36:30 - also maps to interval 23.35 regardless of date 2010-03-18 23:38:21 - also maps to interval 23:35 regardless of date 2010-03-18 23:39:07 - also maps to interval 23:35 regardless of date 2010-03-18 23:41:44 - also maps to interval 23:40 regardless of date 2010-03-18 23:42:13 - also maps to interval 23:40 regardless of date 2010-03-18 23:43:37 - also maps to interval 23:40 regardless of date
For the sake of clarity I’m aiming for this result:
DateTime Interval 2009-02-18 23:32:29 23:30 2009-02-18 23:34:41 23:30 2009-02-18 23:35:40 23.35 2009-02-18 23:39:29 23:35 2009-02-18 23:39:37 23:35 2009-02-18 23:40:14 23:40 2009-02-18 23:43:23 23:40 2009-02-18 23:44:17 23:40 ... 2010-03-18 23:31:19 23:30 2010-03-18 23:33:31 23:30 2010-03-18 23:36:30 23.35 2010-03-18 23:38:21 23:35 2010-03-18 23:39:07 23:35 2010-03-18 23:41:44 23:40 2010-03-18 23:42:13 23:40 2010-03-18 23:43:37 23:40
I’ve read the pandas documentation thoroughly and some questions on here that very loosely relate, but I can’t seem to get anything to achieve the right result.
Update
These are my library and system versions:
Pandas: 0.16.2 Numpy: 1.9.2 System version: '3.4.3 |Anaconda 2.3.0 (x86_64)| (default, Mar 6 2015, 12:07:41) n[GCC 4.2.1 (Apple Inc. build 5577)]
This is the error in full. Here you can see that with my actual data I’m working with a datetime64[ns] Series called question_time.
TypeError Traceback (most recent call last) <ipython-input-416-d5c3256e6b40> in <module>() ----> 1 df_unique['Interval'] = ((df_unique['question_time'] - pd.TimedeltaIndex(df_unique['question_time'].dt.minute % 5, 'm')) - pd.TimedeltaIndex(df_unique['question_time'].dt.second , 's')).dt.time //anaconda/lib/python3.4/site-packages/pandas/core/frame.py in __setitem__(self, key, value) 2125 else: 2126 # set column -> 2127 self._set_item(key, value) 2128 2129 def _setitem_slice(self, key, value): //anaconda/lib/python3.4/site-packages/pandas/core/frame.py in _set_item(self, key, value) 2209 # value exeption to occur first 2210 if len(self): -> 2211 self._check_setitem_copy() 2212 2213 def insert(self, loc, column, value, allow_duplicates=False): //anaconda/lib/python3.4/site-packages/pandas/core/generic.py in _check_setitem_copy(self, stacklevel, t, force) 1302 raise SettingWithCopyError(t) 1303 elif value == 'warn': -> 1304 warnings.warn(t, SettingWithCopyWarning, stacklevel=stacklevel) 1305 1306 def __delitem__(self, key): TypeError: issubclass() arg 2 must be a class or tuple of classes
The issue seems to be with the SettingWithCopyError. I tried resetting all my variables and now I am getting this same warning with another operation as well.
Advertisement
Answer
Not sure of a better method but you can construct 2 TimeDeltaIndices and subtract this from your value, I use modulus op % to calc the number of minutes to subtract:
In [129]:
df['Interval'] = ((df['DateTime'] - pd.TimedeltaIndex(df['DateTime'].dt.minute % 5, 'm')) - pd.TimedeltaIndex(df['DateTime'].dt.second , 's')).dt.time
df
Out[129]:
DateTime Interval
0 2009-02-18 23:32:29 23:30:00
1 2009-02-18 23:34:41 23:30:00
2 2009-02-18 23:35:40 23:35:00
3 2009-02-18 23:39:29 23:35:00
4 2009-02-18 23:39:37 23:35:00
5 2009-02-18 23:40:14 23:40:00
6 2009-02-18 23:43:23 23:40:00
7 2009-02-18 23:44:17 23:40:00
8 2010-03-18 23:31:19 23:30:00
9 2010-03-18 23:33:31 23:30:00
10 2010-03-18 23:36:30 23:35:00
11 2010-03-18 23:38:21 23:35:00
12 2010-03-18 23:39:07 23:35:00
13 2010-03-18 23:41:44 23:40:00
14 2010-03-18 23:42:13 23:40:00
15 2010-03-18 23:43:37 23:40:00