Commit e4294d91 authored by Woude, Auke van der's avatar Woude, Auke van der
Browse files

Added README file

parent d59ade56
Dear CTDAS users,
We hope you’re all doing well! We would like to inform you that we have restructured our CTDAS code base on GIT.
This new structure is completely in python3 and follows the setup of the 7 classes defined in the 2017 GMD paper.
So e.g. instead of having a project-related folders like “TM5” or “STILT”, we now have folders for obsoperators and statevectors.
In the current master code on GIT, we have included baseclasses and the code for the standard CarbonTracker Europe CO2 version (labeled cteco2).
Other previous projects have been moved to a temporary “archive” folder,
from which each of you can take your project specific code and move it into the new structure.
We would like to encourage all users to make use of the GITlab repository as much as possible,
so that we can all benefit from each other’s work there.
Note that moving your code to the new structure is very little work,
it is a matter of moving files to the directories and changing the path for the import of the functions.
We have included further information below on the new structure and how to work with it on GIT.
Please take a look, and let us know in case you have further questions.
We hope that with this step, we can enhance the collaboration between all of us on this code base,
and we will therefore also send you an invitation to our slack channel code-ctdas,
so that we can keep in touch there with CTDAS code related questions.
Kind regards,
Ingrid, Wouter and Auke
===================================================
The new structure
The code has been updated to python3, and the structure is as follows:
The 7 components of the CTDAS system each have their own folder within the da folder.
- cyclecontrol - dasystems
- platform
- statevectors - observations - obsoperators - optimizers
Each of the 7 main folders contains the cteco2 specific code, and in some cases, there is a baseclass module from which project specific code can inherit the properties (e.g. in optimizers). Further folders are:
- pipelines (this contains the pipelines that are called in the main ctdas.py script, currently implemented for the cteco2 setup)
- analysis (here we have standard analysis tools, plus a subfolder for cteco2)
- tools
- preprocessing
- rc
- doc
- archive (this contains all code that was previously in project-folders, this can be
moved into the main new code, further details below).
Furthermore, in the main CTDAS folder, there are the start_ctdas and clone_ctdas shell scripts, and the main ctdas.py, ctdas.jb and ctdas.rc files in the templates folder.
Your project
In case you already had your project specific CTDAS code on GIT, please read this section on how to update to the new structure. New (CTDAS and/or GIT) users can read the sections below. Note that we prefer users to work on branches in the main code, rather than on independent forks, so that we can profit for each other’s developments.
Please follow these steps (two examples below for fork or branch):
1. Create a new temporary branch to make your project-specific code in the new structure
2. Get your changes from your original project-specific branch/fork into the new temporary structure branch
3. Merge request from temporary branch to the master
You can check if you are on the main code or a fork using:
`git remote -v`
In case you are on the main code, you will get something like:
`origin https://git.wur.nl/ctdas/CTDAS.git`
In case you are on a fork, you will get something like:
`origin https://git.wur.nl/woude033/CTDAS.git (fetch)`
`origin https://git.wur.nl/woude033/CTDAS.git (push)`
`upstream https://git.wur.nl/ctdas/CTDAS.git (fetch)`
`upstream https://git.wur.nl/ctdas/CTDAS.git (push)`
You can check on which branch you are as follows:
`git branch`
The output will look like:
` master`
` * feature-STILT`
Example: if you are working on a fork, start at step 0, else, if you are working in a branch of the main code (https://git.wur.nl/ctdas/CTDAS.git) e.g. called feature-STILT start at step 1.
0. In case you are working on a fork:
Make a new folder locally and get a new clone from the master of the main code:
`git clone https://git.wur.nl/ctdas/CTDAS`
1. Make a new temporary branch on the main code:
`git checkout -b temp_stilt_new_structure remotes/origin/master`
2. If you were working in a fork (else skip to step 3): merge your fork as a new temporary branch in the main code.
Go to https://git.wur.nl/ctdas/CTDAS and go to ‘merge requests’, and ‘click new merge request’. On the left, fill in your fork and on the right the main CTDAS code and your new temporary branch temp_stilt_new_structure.
3. Copy a file from your previous branch to the new temporary branch:
`git checkout feature-STILT stilt/optimizer.py`
4. Move this file to the correct place in the new structure:
`git mv stilt/optimizer.py optimizers/optimizer_stilt.py`
5. Make sure everything is working with this new code, and then:
`git add .`
`git commit -m “Added STILT optimizer” git push`
6. After you finish to implement and test all your changes: Go to https://git.wur.nl/ctdas/CTDAS and go to ‘merge requests’, and ‘click new merge request’.
====================================================
Some useful GIT commands:
1. Check if you’re on the master branch:
`git branch`
2. If you’re not on the master branch, switch to it:
`git checkout master`
3. Switching to another existing branch:
`git checkout feature-STILT`
4. If you want to compare your code to the master:
`git fetch`
`git status`
5. To get the changes from the remote master:
`git pull`
6. Moving a file:
`git mv stilt/optimizer.py optimizers/optimizer_stilt.py`
7. Committing changes:
`git add .`
`git commit -m “A message” git push`
8. If you want to replace your local master code with the remote master code, follow
these steps (warning! Your previous version will be lost!): git fetch origin
`git reset --hard origin/master`
#### Experienced CTDAS users, but new to GIT:
Follow the first steps for new users (see below) to get your copy of the current CTDAS code from GIT in a new folder. Then compare the changes in your local code and merge your changes into the new structure (see description above) in your temporary branch. It might happen that you want to create a copy of an existing file on GIT, that you adapt to your project. There is no git cp command, so to create a copy with GIT history, follow these
steps:
`cp optimizers/optimizer_cteco2.py optimizers/optimizer_ctesamco2.py git add optimizers/optimizer_ctesamco2.py`
`git commit -m “Added CTE South America CO2 optimizer”`
`git push`
##### Python3
To update your code to python3, the tool 2to3 is very convenient. See for more details:
https://docs.python.org/3/library/2to3.html
New CTDAS users
Make a folder, then:
`git clone https://git.wur.nl/ctdas/CTDAS`
Make your temporary branch to develop your code:
`git checkout -b temp_stilt_new_structure remotes/origin/master`
(Later on, you can do a merge request to merge your changes into the master branch).
The main code is found in the da folder. You will find subfolders for each of the components
of the CTDAS system (see information above about the structure). Furthermore, we have a
script to start a new run, as follows:
`./start_ctdas.sh /somedirectory/ name_of_your_run`
This will create a new folder for your run, and makes a copy of the da folder. Furthermore, the main ctdas.py will be created, named name_of_your_run.py, along with the main jb and rc file. These are based on the templates in the template folder. In this way, you will always have the current version of the code with your run. For further information, please refer to: https://www.carbontracker.eu/ctdas/
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment