Workflow quick reference

From Charm-Tau Detector
(Difference between revisions)
Jump to: navigation, search
(Created page with "0. Зарегистрироваться на BINP/GCF кластере зайти на stark или proxima Если ещё нет создать ssh ключ. зайти в...")
 
(→‎Build and run)
 
(17 intermediate revisions by 5 users not shown)
Line 1: Line 1:
−
0. Зарегистрироваться на BINP/GCF кластере
+
== Register and prepare account ==
 +
* Register at BINP/GCF cluster. Address to [[User:A.M.Suharev|Andrey Sukharev]] or [[User:D.A.Maksimov‏|Dmitry Maximov]]
  
−
зайти на stark или proxima
+
* Log in to <code>stark.inp.nsk.su</code> (BINP local access only) or <code>proxima.inp.nsk.su</code> (accessible from the Internet, pubkey authentication only)
−
Если ещё нет создать ssh ключ.
+
  
−
зайти в gitlab https://git.inp.nsk.su/
+
The login servers have similar configuration and share common <code>/home</code>.
−
зарегистрировать этот ключ в своём аккаунте
+
  
−
этот шаг у вас должен быть уже пройден.
+
The <code>git</code> is accessible using <code>ssh</code> protocol with key authorization.
  
−
1. Открыть центральный репозиторий
+
* If necessary, create <code>ssh</code> key using
−
https://git.inp.nsk.su/sctau/aurora
+
ssh-keygen
  
−
и сделать форк к себе.
+
Agree to everything, passwordless keys are allowed.
  
−
2. Настройка рабочей среды.
+
Then you'll have in your <code>~/.ssh/</code> two files <code>id_rsa</code> and <code>id_rsa.pub</code> - these are your private and public keys.
−
# создадим рабочую директорию
+
Full paths to the files are displayed in the terminal.
−
mkdir workarea
+
−
cd workarea
+
  
−
# Директории для сборки и запуска
+
* Log in to <code>gitlab</code> server https://git.inp.nsk.su/ using the same name/password as for <code>stark</code>
−
mkdir build run
+
* Register your public key for your account: i. e add ~/.ssh/id_rsa.pub contents to a from at the following link
 +
  https://git.inp.nsk.su/-/profile/keys
  
  
−
# Настройка самой базовой среды,
+
== Setup remote ssh connection via PuTTy on Windows ==
−
# данную команду необходимо выполнять каждый раз при входе
+
−
setupSCTAU
+
  
−
# Выберем релиз и его версию, в которой будем работать
+
* Run PuTTy
−
# Для работы требующей стабильности окружения, например физический анализ,  
+
* In the Session tab, in the "Host Name (or IP addres)" field, write: user_name@proxima.nsk.su
−
нужно использовать этот вариант
+
  Use PuTTyGen program to create public and private key
−
asetup SСTauSim,0.1.0
+
  The public key must be sent to [[User:A.M.Suharev|Andrey Sukharev]]
 +
* In the Connection -> SSH -> Auth tab, in the "Private key file for authentication" field, write the path to your private key
 +
* In the Session tab in the "Saved session" field, write: a name that will be used for the current settings
 +
* In the Session tab, click on the "Save" button
  
−
# Для работ по разработке программных компонент этот
+
== Create working repository (fork) ==
−
asetup SCTauSim,0.1,latest
+
Open central repository
 +
https://git.inp.nsk.su/sctau/aurora
  
 +
and make fork.
  
−
# Для простого запуска готовых примеров этих шагов достаточно
 
−
cd run
 
  
−
# запуск первичных генераторов моделирования
+
== Tune working environment ==
−
ctaurun GenExamples/evtgen.py
+
Each time you log in to a server <code>stark</code>/<code>proxima</code>, you need to
 +
setupSCTAU
 +
asetup ''The software version''
  
−
# Запуск полного моделирования
+
Available version are at least:
−
#
+
* 0.2.3 ``release'' - the environment before first official release
−
# Перед запуском полного моделирования в текущей директории надо положить файл
+
asetup Aurora,0.2.3
−
# taumugamma.root, с входными данными - частицы из первичного генератора
+
* 1.0.0 release - fixed build for detector investigation tasks, for physics and
−
#
+
other task requiring stable environment. The 1.0.X series are intended for bug fixes.
−
# Этот файл можно взять, например, у Виталия
+
asetup Aurora,1.0.0
−
# /home/vvorob/public/tuples/fccedm/taumugamma.root
+
* master - branch and build for future development without any warranty on
 +
stability, compatibility or meaningful working.
 +
  asetup Aurora,master,latest
  
−
ctaurun G4SimExamples/fullsim_example.py
+
Ask software coordinators if you're unsure what to put as ''the software version''.
  
 +
To tune git do once:
  
−
3. Для разработки нового или модификации существующего кода нужны следующие
+
First, do:
−
действия
+
  git sctau init-config
−
возвращаемся в workarea
+
−
# Подготовка рабочей директории (делается один раз)
+
−
git sctau init-workdir ssh://git@git.inp.nsk.su/sctau/aurora.git
+
−
cd aurora
+
  
−
# Получение обновлений с головного репозитория
+
Check if the settings are correct. But the defaults should be fine.
−
# нужно делать периодически при длительном существовании рабочей директории и
+
−
# существенных изменениях в головном
+
−
git fetch upstream
+
  
−
# Подготовка рабочей тематической ветки, эта ветка будет видна другим людям,
+
Then apply the settings:
−
# поэтому название стоит выбирать говорящим и осмысленным
+
git sctau init-config --apply
−
git checkout -b MyDevelopmentBranch upstream/0.1 --no-track
+
  
−
# Если хотим модифицировать существующий пакет, то
+
== tune working directory ==
−
# Добавим пакеты из репозитория
+
To develop new code or to modify existing one, do:
−
git sctau addpkg GenExamples
+
−
и/или
+
−
git sctau addpkg G4SimExamples
+
  
−
# Если надо создать новый пакет, то надо создать всю структуру директорий где
+
create workare in your home directory (at <code>stark</code>/<code>proxima</code>)
−
он должен лежать, написать CMakeLists.txt и всё остальное что нужно для нового
+
  mkdir workarea
−
пакета.
+
cd workarea
  
−
# Сборка
+
create directories for build and run:
−
cd ../build/
+
mkdir build run
−
cmake ../aurora/Projects/WorkDir
+
−
make
+
  
−
# Настройка локального окружения
+
workarea preparation (just once):
−
# эта строчка принципиально важна, чтобы использовались локально собранные
+
git sctau init-workdir ssh://git@git.inp.nsk.su/sctau/aurora.git
−
# пакеты вместо тех, что в релизе
+
−
source x86_64-slc7-gcc7-opt/setup.sh
+
  
−
# Запуск
+
Go to working directory
−
cd ../run
+
cd aurora
−
ctaurun GenExamples/evtgen.py
+
−
ctaurun G4SimExamples/fullsim_example.py
+
  
−
[[Category:Not_public]]
+
Fetch updates from head repository (must be done before '''creating a branch''' when the workarea exists for a long time)
 +
git fetch upstream
 +
 
 +
Create a working branch. Give it a sensible name:
 +
git checkout -b <TopicDevelopmentBranch> upstream/<target_branch> --no-track
 +
 
 +
 
 +
'''The line above will not work if you simply copy-paste it. It is intentional. It is important to choose correct target branch. If unsure ask software coordinators.'''
 +
 
 +
 
 +
To modify an existing package check it out:
 +
git sctau addpkg GenExamples
 +
 
 +
To create new package, one should create its whole directory structure, provide CMakeLists.txt and everything.
 +
 
 +
''Before creating new package ask software coordinators how to name and where to place it''
 +
 
 +
Useful commands to manage packages in you working area:
 +
 
 +
List local packages
 +
git sctau listpkg
 +
 
 +
List packages in the whole repository
 +
git sctau listpkg --all
 +
or with regexp filter
 +
git sctau listpkg --all 'Det'
 +
git sctau listpkg --all '/G4.*U'
 +
 
 +
 
 +
Remove local package (the repository left intact)
 +
git sctau rmpkg <PackageName>
 +
 
 +
It is recommended to keep locally only the packages under active development.
 +
 
 +
== Build and run ==
 +
To build:
 +
cd ../build/
 +
cmake ../aurora/Projects/WorkDir
 +
make
 +
 
 +
To setup local environment (to use locally built packages instead of default versions):
 +
source x86_64-slc7-gcc9-opt/setup.sh
 +
 
 +
To run
 +
cd ../run
 +
 
 +
Run primary generators:
 +
ctaurun GenExamples/evtgen.py
 +
 
 +
Run full simulation:
 +
ctaurun G4SimExamples/fullsim_example.py
 +
 
 +
Download standard job options and run with local file:
 +
get_joboptions fullsim_example.py
 +
ctaurun ./fullsim_example.py
 +
 
 +
== Commit changes ==
 +
In the 'aurora' directory:
 +
 
 +
Add changed files
 +
git add <list of files>
 +
 
 +
remove unnecessary files
 +
git rm <list of files>
 +
 
 +
 
 +
Commit changes in the local repository:
 +
git commit -m 'Meaningful message concerning the introduced changes'
 +
 
 +
Each commit should contain a minimal set of logically interconnected changes.
 +
You may (and will) have many commits when developing a package.
 +
 
 +
Put the changes to the server:
 +
git push
 +
 
 +
Wheh doing push for the first time, it is necessary to do it like
 +
git push --set-upstream origin <TopicDevelopmentBranch>
 +
 
 +
 
 +
After the push, <code>git</code> displays a link to be followed for a <code>Merge Request</code> (adding your contributions to common repository).
 +
 
 +
When doing a merge request, make sure:
 +
* the branch is correct
 +
* changes are well described
 +
** reasons for the changes are demonstrated
 +
** the changes are described
 +
** influence on the other software and possible side effects are indicated
 +
** there are links to related issues, if any
 +
 
 +
== Running graphical applications remotely ==
 +
If your connection does not allow you to run X applications directly, please try [[x2go]].
 +
 
 +
 
 +
[[Category:Software]]

Latest revision as of 12:41, 15 February 2024

Contents

[edit] Register and prepare account

  • Log in to stark.inp.nsk.su (BINP local access only) or proxima.inp.nsk.su (accessible from the Internet, pubkey authentication only)

The login servers have similar configuration and share common /home.

The git is accessible using ssh protocol with key authorization.

  • If necessary, create ssh key using 
ssh-keygen

Agree to everything, passwordless keys are allowed.

Then you'll have in your ~/.ssh/ two files id_rsa and id_rsa.pub - these are your private and public keys. Full paths to the files are displayed in the terminal.

  • Log in to gitlab server https://git.inp.nsk.su/ using the same name/password as for stark
  • Register your public key for your account: i. e add ~/.ssh/id_rsa.pub contents to a from at the following link 
https://git.inp.nsk.su/-/profile/keys


[edit] Setup remote ssh connection via PuTTy on Windows

  • Run PuTTy
  • In the Session tab, in the "Host Name (or IP addres)" field, write: user_name@proxima.nsk.su 
 Use PuTTyGen program to create public and private key
 The public key must be sent to Andrey Sukharev
  • In the Connection -> SSH -> Auth tab, in the "Private key file for authentication" field, write the path to your private key
  • In the Session tab in the "Saved session" field, write: a name that will be used for the current settings
  • In the Session tab, click on the "Save" button

[edit] Create working repository (fork)

Open central repository 

https://git.inp.nsk.su/sctau/aurora

and make fork.


[edit] Tune working environment

Each time you log in to a server stark/proxima, you need to 

setupSCTAU
asetup The software version

Available version are at least:

  • 0.2.3 ``release - the environment before first official release 
asetup Aurora,0.2.3
  • 1.0.0 release - fixed build for detector investigation tasks, for physics and

other task requiring stable environment. The 1.0.X series are intended for bug fixes. 

asetup Aurora,1.0.0
  • master - branch and build for future development without any warranty on

stability, compatibility or meaningful working. 

asetup Aurora,master,latest

Ask software coordinators if you're unsure what to put as the software version.

To tune git do once:

First, do: 

git sctau init-config

Check if the settings are correct. But the defaults should be fine.

Then apply the settings: 

git sctau init-config --apply

[edit] tune working directory

To develop new code or to modify existing one, do:

create workare in your home directory (at stark/proxima) 

mkdir workarea
cd workarea

create directories for build and run: 

mkdir build  run

workarea preparation (just once): 

git sctau init-workdir ssh://git@git.inp.nsk.su/sctau/aurora.git

Go to working directory 

cd aurora

Fetch updates from head repository (must be done before creating a branch when the workarea exists for a long time) 

git fetch upstream

Create a working branch. Give it a sensible name: 

git checkout -b <TopicDevelopmentBranch> upstream/<target_branch> --no-track


The line above will not work if you simply copy-paste it. It is intentional. It is important to choose correct target branch. If unsure ask software coordinators.


To modify an existing package check it out: 

git sctau addpkg GenExamples

To create new package, one should create its whole directory structure, provide CMakeLists.txt and everything. 

Before creating new package ask software coordinators how to name and where to place it

Useful commands to manage packages in you working area:

List local packages 

git sctau listpkg

List packages in the whole repository 

git sctau listpkg --all

or with regexp filter 

git sctau listpkg --all 'Det'
git sctau listpkg --all '/G4.*U'


Remove local package (the repository left intact) 

git sctau rmpkg <PackageName>

It is recommended to keep locally only the packages under active development.

[edit] Build and run

To build: 

cd ../build/
cmake ../aurora/Projects/WorkDir
make

To setup local environment (to use locally built packages instead of default versions): 

source x86_64-slc7-gcc9-opt/setup.sh

To run 

cd ../run

Run primary generators: 

ctaurun GenExamples/evtgen.py

Run full simulation: 

ctaurun G4SimExamples/fullsim_example.py

Download standard job options and run with local file: 

get_joboptions fullsim_example.py
ctaurun ./fullsim_example.py

[edit] Commit changes

In the 'aurora' directory:

Add changed files 

git add <list of files>

remove unnecessary files 

git rm <list of files>


Commit changes in the local repository: 

git commit -m 'Meaningful message concerning the introduced changes'

Each commit should contain a minimal set of logically interconnected changes. You may (and will) have many commits when developing a package.

Put the changes to the server: 

git push

Wheh doing push for the first time, it is necessary to do it like 

git push --set-upstream origin <TopicDevelopmentBranch>


After the push, git displays a link to be followed for a Merge Request (adding your contributions to common repository).

When doing a merge request, make sure:

  • the branch is correct
  • changes are well described
    • reasons for the changes are demonstrated
    • the changes are described
    • influence on the other software and possible side effects are indicated
    • there are links to related issues, if any

[edit] Running graphical applications remotely

If your connection does not allow you to run X applications directly, please try x2go.

Retrieved from "https://ctd.inp.nsk.su/wiki/index.php?title=Workflow_quick_reference&oldid=1386"
Personal tools