Implement currency data update workflow#61
Implement currency data update workflow#61Wegatriespython wants to merge 6 commits intoIAMconsortium:mainfrom
Conversation
|
From the description this is great, thanks for the contribution! One knee-jerk question prior to a real review: the 25 added data files each have 1 data point in them. Did you consider / would it be worse/better to follow an approach similar to the generation of emissions.py? IOW, generate a Python source file containing a data structure with all relevant data. This could be more compact. |
|
wow @Wegatriespython didn't expect you to address this within 2 hours. Thank you so much! ;) |
|
Thanks @Wegatriespython for the PR. @khaeru, do you have time to review? |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #61 +/- ##
==========================================
- Coverage 96.07% 87.97% -8.11%
==========================================
Files 5 5
Lines 204 291 +87
==========================================
+ Hits 196 256 +60
- Misses 8 35 +27 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
I was waiting/hoping to discuss the question I asked above. |
|
@khaeru Hey Paul, the 1 file per method period was following the spec in the update stub. It does look quite ugly agreed, but I think if we were to consolidate, 1 txt file will do just as well for brevity. I don't see the added value of making it a python file, unless I'm missing something |
OK, thanks for clarifying. I guess the lesson is that a stub is a stub because it's only half-conceived, and should be taken with a grain of salt/viewed critically.
Two things:
Having written this out, I think I'd prefer this approach, but if you are time-limited please just let me know and I'll add a commit or two to the branch to make the change. |
|
I see, I can pick this up but it would be around EOD. Does that work? |
|
Yes, from my POV there's no hurry. Thanks! |
Follow emissions pattern
|
@khaeru Done! As requested there is now a single generated Python module ( |
This replaces the stubbed
iam_units.update.currency()with a working generator and expands currency support from one hardcoded entry (EXC, EUR, 2005) to all five OECD Table 4 methods (EXC,EXCE,PPPGDP,PPPPRC,PPPP41) across five periods (2005, 2010, 2015, 2020, 2024) for EUR. This partly addresses #25.The original stub in
update.pyspecified three requirements for an implementation:sdmx1to query either the World Bank or OECD SDMX API.iam_units/data/currency/, one file per (method, period).The inline NB comments in
currency.pynoted two extensions: loading from file instead of embedded constants, and extending to cover other currencies. This PR addresses the first; EUR remains the only non-USD currency.The current PR:
currency()in the update submodule usingsdmx1to query OECD Table 4 (DSD_NAMAIN10@DF_TABLE4), following the pattern established for emissions data.iam_units/data/currency/, committed as package data and loaded at runtime without any dependency onsdmx1.configure_currency()against silent wrong answers when called twice with different methods for the same currency/period pair — now raisesValueError. Same-method calls are idempotent.DEVELOPING.rstand the expanded capabilities in the README.EUR rows are derived from the DEU series in Table 4. For exchange rates this is exact; for PPP methods the values are Germany-specific. This follows the approach in #60 which switched EUR deflators to Eurozone-wide values, but PPP is inherently country-level — the choice is documented in code and
DEVELOPING.rst.