解决方案:zipline24/7导入bundle出错KeyError: Timestamp(‘2019-09-09 23:57:00+0000′, tz=’UTC’)
错误如下:
(envs_zipline) C:\Users\>zipline ingest -b forex-csvdir-bundle Loading custom pricing data: [##################------------------] 50% | EURUSD: sid 0 Merging minute equity files: [------------------------------------] 0 Traceback (most recent call last): File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\pandas\core\indexes\base.py", line 2393, in get_loc return self._engine.get_loc(key) File "pandas\_libs\index.pyx", line 436, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:10160) File "pandas\_libs\index.pyx", line 455, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:9683) KeyError: Timestamp('2019-09-09 23:57:00+0000', tz='UTC') During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\Users\Anaconda3\envs\envs_zipline\Scripts\zipline-script.py", line 11, in <module> load_entry_point('zipline==1.3.0', 'console_scripts', 'zipline')() File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\click\core.py", line 722, in __call__ return self.main(*args, **kwargs) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\click\core.py", line 697, in main rv = self.invoke(ctx) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\click\core.py", line 1066, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\click\core.py", line 895, in invoke return ctx.invoke(self.callback, **ctx.params) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\click\core.py", line 535, in invoke return callback(*args, **kwargs) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\__main__.py", line 348, in ingest show_progress, File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\bundles\core.py", line 451, in ingest pth.data_path([name, timestr], environ=environ), File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\bundles\csvdir.py", line 94, in ingest self.csvdir) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\bundles\csvdir.py", line 156, in csvdir_bundle show_progress=show_progress) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\minute_bars.py", line 697, in write write_sid(*e, invalid_data_behavior=invalid_data_behavior) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\minute_bars.py", line 730, in write_sid self._write_cols(sid, dts, cols, invalid_data_behavior) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\minute_bars.py", line 810, in _write_cols latest_min_count = all_minutes.get_loc(last_minute_to_write) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\pandas\core\indexes\datetimes.py", line 1418, in get_loc return Index.get_loc(self, key, method, tolerance) File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\pandas\core\indexes\base.py", line 2395, in get_loc return self._engine.get_loc(self._maybe_cast_indexer(key)) File "pandas\_libs\index.pyx", line 436, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:10160) File "pandas\_libs\index.pyx", line 455, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:9683) KeyError: Timestamp('2019-09-09 23:57:00+0000', tz='UTC')
有时候错误是下面的显示:
(envs_zipline) C:\Users\>zipline ingest -b forex-csvdir-bundle Loading custom pricing data: [##################------------------] 50% | EURUSD: sid 0 Merging minute equity files: [------------------------------------] 0 Traceback (most recent call last): File "pandas\_libs\index.pyx", line 464, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:9810) File "pandas\_libs\hashtable_class_helper.pxi", line 756, in pandas._libs.hashtable.Int64HashTable.get_item (pandas\_libs\hashtable.c:13913) File "pandas\_libs\hashtable_class_helper.pxi", line 762, in pandas._libs.hashtable.Int64HashTable.get_item (pandas\_libs\hashtable.c:13857) KeyError: 1568073420000000000 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\pandas\core\indexes\base.py", line 2393, in get_loc return self._engine.get_loc(key) File "pandas\_libs\index.pyx", line 436, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:10160) File "pandas\_libs\index.pyx", line 466, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:9897) KeyError: Timestamp('2019-09-09 23:57:00+0000', tz='UTC')
两种情况,都是一样的,类似的问题。
zipline网上都说是文档最多的回测平台,或许是吧,可是官方的文档写的真是非常的差,其中太多的坑。难道是有意的,这样社区才有热度?
本来是想改为适合外汇历史数据的bundle和 TradingCalendaryy就是我们要修改的重点,这样才能导入我们自己的数据,zipline在数据规范上做的太多,个人觉得这就是最大的坑,特别对于外汇市场来说,各平台数据和开盘时间,相差不少。
为了在数据上不出错,我们选择7*24小时的交易日历。其实他已经自带了,可是官方就是什么也不写,自己找吧,搞的每个人都去研究他的源码。
下面给出正确的文件源码,每一行都是必要的。
解决方案
自定义bundle
在.zipline/extension.py中写入
import pandas as pd import datetime from zipline.data.bundles import register from zipline.data.bundles.csvdir import csvdir_equities start_session = pd.Timestamp('2000-01-01', tz='utc') end_session = pd.Timestamp(datetime.datetime.now().strftime('%Y-%m-%d'), tz='utc') register( 'forex-csvdir-bundle', csvdir_equities( ['minute'], #daily,minute 'D:/zipline/', ), calendar_name='FOREX', # NYSE,FOREX start_session=start_session, end_session=end_session, minutes_per_day=1440 #这就坑,交易日历会根据这个生成时间表,不写的话默认390 )
在环境路径中找到envs_zipline\Lib\site-packages\trading_calendars\calendar_utils.py,在_default_calendar_aliases中新增一项自定义的bundles名称与calendar的对应关系
_default_calendar_aliases = { 'FOREX': '24/7',# 新增这个bundles名称与calendar的对应关系,24/7这个是默认就有的。 'NYSE': 'XNYS', 'NASDAQ': 'XNYS', 'BATS': 'XNYS', 'FWB': 'XFRA', 'LSE': 'XLON', 'TSX': 'XTSE', 'BMF': 'BVMF', 'CME': 'CMES', 'CBOT': 'CMES', 'COMEX': 'CMES', 'NYMEX': 'CMES', 'ICE': 'IEPA', 'ICEUS': 'IEPA', 'NYFE': 'IEPA', 'CFE': 'XCBF', }
按你说的改了,还是不行 快自闭了