Tsarin Kwamfuta da Ajiya na Haɗin Kai don PUNCH4NFDI

1. Gabatarwa

PUNCH4NFDI (Barbashi, Sararin Samaniya, Ƙwayoyin Nukiliya da Hadrons don Tsarin Bayanan Bincike na Ƙasa) babbar ƙungiya ce ta Jamus da DFG (Deutsche Forschungsgemeinschaft) ta ba da kuɗi. Tana wakiltar kimanin masana kimiyya 9,000 daga fagen barbashi, taurari, barbashin taurari, hadron, da ilimin nukiliya. Babban manufar ƙungiyar ita ce kafa dandalin bayanan kimiyya na haɗin kai, mai bin ƙa'idodin FAIR (Ana iya Samuwa, Ana iya Kaiwa, Ana iya Haɗa Kai, Ana iya sake Amfani da su). Wannan gudummawar ta yi cikakken bayani game da ra'ayoyin tsarin gine-gine—Compute4PUNCH da Storage4PUNCH—waɗanda aka ƙera don haɗa hanyoyin samun damar albarkatun kwamfuta masu bambanta (HPC, HTC, Girgije) da albarkatun ajiya waɗanda cibiyoyin membobin ke bayarwa a duk faɗin Jamus.

2. Tsarin Kwamfuta na Haɗin Kai – Compute4PUNCH

Ƙaddamarwar Compute4PUNCH tana magance ƙalubalen samar da damar shiga cikin sauƙi ga tarin albarkatun kwamfuta da ake da su ba tare da sanya manyan canje-canje ga tsarin aiki na masu ba da albarkatu ba.

3. Tsarin Ajiya na Haɗin Kai – Storage4PUNCH

Storage4PUNCH yana nufin haɗa tsarin ajiya na al'umma, waɗanda suka fi dogara da fasahohin dCache da XRootD, waɗanda suka kafu sosai a cikin Kimiyyar Makamashi Mai Girma (HEP).

4. Cikakkun Bayanai na Fasaha & Tsarin Lissafi

Ana iya ƙirƙira ainihin dabaru na tsarawa a cikin COBalD/TARDIS a matsayin matsala ta ingantawa. Bari $R = \{r_1, r_2, ..., r_n\}$ ya zama tarin buƙatun albarkatu daga tarin HTCondor, kuma $B = \{b_1, b_2, ..., b_m\}$ ya zama tarin nau'ikan albarkatun baya da ake da su (misali, HPC node, Cloud VM). Kowane buƙata $r_i$ yana da buƙatu (cores, ƙwaƙwalwar ajiya, software). Kowane tushen baya $b_j$ yana da aikin farashi $C_j(r_i)$ da lokacin samarwa $T_j(r_i)$.

Manufar meta-scheduler ita ce nemo taswira $M: R \rightarrow B$ wanda ke rage aikin farashi gabaɗaya, sau da yawa jimlar farashin kuɗi da lokacin kammalawa, bisa ga ƙuntatawa kamar ƙididdiga na baya da samuwar software:

$$\min_{M} \sum_{r_i \in R} \left[ \alpha \cdot C_{M(r_i)}(r_i) + \beta \cdot T_{M(r_i)}(r_i) \right]$$

inda $\alpha$ da $\beta$ su ne abubuwan ma'auni. Wannan yana tsara ƙalubalen "haɗin kai mai ƙarfi kuma mai bayyana".

5. Sakamakon Ƙirar Farko & Aiki

Takardar ta ba da rahoto game da farkon gogewa tare da aikace-aikacen kimiyya da ke gudana akan ƙirar farko da ake da su. Duk da yake ba a yi cikakken bayani game da ƙididdiga na musamman a cikin abin da aka ba da shi ba, amma nasarar aiwatarwa tana nuna:

• Haɗin Aiki: Tarin HTCondor/COBalD/TARDIS ya yi nasarar tura ayyuka zuwa tsarin baya daban-daban (HTC, HPC, Girgije).
• Isar da Software: CVMFS da kwantena sun samar da yanayin software da ake buƙata a cikin nodes masu aiki daban-daban.
• Shiga Mai Amfani: JupyterHub da nodes na shiga sun yi aiki azaman hanyoyin shiga masu inganci ga masu bincike.

Zane na Ra'ayi: Ana iya ganin tsarin tsarin a matsayin samfurin mataki uku:

1. Matakin Shiga Mai Amfani: JupyterHub, Nodes na Shiga, Token AAI.
2. Matakin Haɗin Kai & Tsarawa: Tarin HTCondor + COBalD/TARDIS Meta-scheduler.
3. Matakin Albarkatu: Tushen baya daban-daban (HPC clusters, HTC farms, Cloud VMs) da ajiya na haɗin kai (dCache, XRootD instances).

6. Tsarin Bincike: Misalin Amfani

Misali: Mai binciken kimiyyar nukiliya yana buƙatar sarrafa ayyukan siminti na Monte Carlo 10,000, kowanne yana buƙatar cores na CPU 4, RAM 16 GB, da takamaiman tarin software (Geant4, ROOT).

1. Ƙaddamarwa: Mai binciken ya shiga cikin PUNCH JupyterHub, ya rubuta rubutun bincike, ya ƙaddamar da ayyuka 10,000 zuwa tsarin tsarawa na HTCondor na gida.
2. Meta-Tsarawa: COBalD/TARDIS yana lura da jerin HTCondor. Yana kimanta tushen baya da ake da su: HTC farm na Jami'ar A (farashi ƙasa, lokacin jerin gwano mai yawa), HPC cluster na Cibiyar B (farashi matsakaici, kayan aiki na musamman), da girgije na kasuwanci (farashi mai yawa, samuwa nan take).
3. Yanke Shawara & Ai: Ta amfani da samfurin farashinsa, TARDIS na iya yanke shawarar ƙaddamar da ayyuka 2,000 nan take zuwa girgije don farawa da sauri, yayin da yake fitar da sauran akan HTC farm mai arha. Yana amfani da token AAI don tabbatar da asali akan duk tsarin.
4. Software & Bayanai: Kowane aiki, ko da kuwa tushen baya, yana jawo yanayin Geant4/ROOT daga CVMFS. Ana karɓar bayanan shigarwa daga sunan sarari na Storage4PUNCH na haɗin kai (misali, ta hanyar XRootD), kuma ana sake rubuta sakamako zuwa ƙarshen ajiya da aka keɓe.
5. Kammalawa: Mai binciken yana lura da tarawa da sakamako daga jerin ayyukan HTCondor guda ɗaya, ba tare da sanin aiwatar da tsarin ƙasa da yawa ba.

7. Bincike Mai Zurfi & Ra'ayi na Kwararru

Babban Fahimta: PUNCH4NFDI ba yana gina wani girgije ba; yana ƙera matakin haɗin kai na siyasa da fasaha mai ma'ana. Ainihin ƙirƙirarsa ta ta'allaka ne a cikin meta-scheduler na COBalD/TARDIS, wanda ke aiki azaman "mai fassara na diflomasiyya" don raba albarkatu, ba mai haɗa kai ba. Wannan yana yarda da ikon mallakar clusters na cibiyoyin da ake da su—gaskiyar da ba za a iya sasantawa ba a cikin ilimin Jamus—yayin da har yanzu yana ƙirƙirar babban albarkatu mai aiki.

Kwararar Hankali: Hankali ba shi da lahani: fara da mai amfani (JupyterHub/login), ɓoye hargitsi ta hanyar tsarin tsarawa mai gwaji (HTCondor), sannan a yi amfani da dillali mai hikima (TARDIS) don taswirar buƙatun zuwa tushen baya na zahiri, mai yuwuwar siyasa. Dogaro akan CVMFS da kwantena don software wani babban nasara ne, yana magance matsalar "jahim dogaro" da ke addabar yawancin haɗin kai. Dabarar ajiya tana da hankali sosai, tana gina kan ingantaccen duo na dCache/XRootD daga HEP, tana guje wa rikicin ƙoƙarin tilasta sabuwar fasaha ɗaya.

Ƙarfi & Kurakurai:

• Ƙarfi: Ƙanƙantar mamayewa shine babban ikonsa. Ba ya buƙatar masu bayarwa su canza manufofin gida. Amfani da kayan aiki masu girma, waɗanda al'umma ke tafiyar da su (HTCondor, CVMFS, dCache) yana rage haɗari sosai kuma yana ƙara dorewa, sabanin ayyukan da aka gina akan tsararrun tsare-tsare. Mayar da hankali kan ƙa'idodin FAIR ya yi daidai da umarnin tallafin zamani.
• Kurakurai & Hadari: Hanyar meta-scheduler tana gabatar da matsalar rikitarwa guda ɗaya da yuwuwar gazawa. COBalD/TARDIS, duk da yana da alƙawari, ba shi da ƙarfi kamar sauran sassan. "Ƙididdiga" na fasahar ajiya/metadata (kamar Rucio) yana nuna alamar matsala mafi wahala tana gaba: sarrafa bayanai cikin hikima. Idan ba tare da shi ba, wannan haɗin kai ne na kwamfuta tare da kundayen ajiya, ba dandali mai mayar da hankali kan bayanai ba. Akwai kuma haɗarin rashin iya hasashen aiki ga masu amfani, yayin da ayyukansu ke tsallaka tsakanin tsarin gine-gine daban-daban.

Hanyoyin Aiki masu Amfani:

1. Ga Masu Gina PUNCH: Ƙara ƙarfi don sanya TARDIS ya zama mai ƙarfi kuma a iya gani. Ma'auni da rajistan yanke shawara su ne zinariya don ingantawa da gina amana. Ba da fifiko ga haɗa matakin sarrafa bayanai (kamar Rucio) na gaba; kwamfuta ba tare da bayanai masu hikima ba rabin mafita ne.
2. Ga Sauran Ƙungiyoyi: Wannan zane ne da ya cancanci a yi koyi da shi, musamman falsafar "haɗin kai akan maye gurbin". Duk da haka, kimanta ko al'ummarku tana da daidai da CVMFS—idan ba haka ba, wannan shine yanke shawararku ta farko ta gini/siya.
3. Ga Masu Bayar da Albarkatu: Wannan samfurin yana da haɗari ƙasa gare ku. Ku shiga cikinsa. AAI mai amfani da alama hanya ce mai tsabta don ba da damar shiga ba tare da lalata tsaron gida ba. Wannan riba ce ga ganuwa da amfani.

8. Ayyukan Gaba & Taswirar Ci Gaba

Tsarin PUNCH4NFDI yana shimfiɗa tushe don ayyuka da yawa na ci gaba da hanyoyin bincike:

• Hanyoyin Aiki na Fage Daban-daban: Ba da damar ingantattun hanyoyin bincike masu matakai da yawa waɗanda ke motsawa cikin sauƙi tsakanin siminti (HPC), sarrafa abubuwan da suka faru masu yawan aiki (HTC), da horar da injin koyo (Cloud GPUs).
• Tsarawa Mai Mayar da Hankali kan Bayanai: Haɗa haɗin ajiya sosai tare da mai tsara kwamfuta. Siffofin COBald/TARDIS na gaba za su iya haɗa wurin bayanai (rage canja wurin WAN) da shirye-shiryen farko cikin aikin farashinsa, suna matsawa zuwa tsarawa mai sanin bayanai.
• Haɗin Kai tare da Ma'ajiyar Bayanai na FAIR: Yin aiki azaman ginshiƙin kwamfuta mai ƙarfi don ma'ajiyar bayanan FAIR na ƙasa, yana ba masu bincike damar bincika manyan bayanan kai tsaye inda aka ajiye su, bin tsarin "kwamfuta-zuwa-bayanai".
• AI/ML azaman Sabis: Za a iya faɗaɗa fuskar JupyterHub da tushen baya mai iya faɗaɗawa tare da ingantattun muhalli don tsararrun tsarin AI/ML na musamman (PyTorch, TensorFlow) da samun damar albarkatun GPU, yana ba da damar AI ga kimiyyar zahiri.
• Faɗaɗawa zuwa Albarkatun Ƙasashen Duniya: Za a iya faɗaɗa samfurin haɗin kai don haɗa albarkatu daga ƙaddamarwar Turai kamar European Open Science Cloud (EOSC) ko wuraren kwamfuta na LHC (WLCG), ƙirƙirar ainihin tsarin bincike na Turai.

Taswirar ci gaba mai yiwuwa ta haɗa da ƙarfafa ƙirar farko, ƙididdige adadin albarkatun da aka haɗa, aiwatar da mafita na ƙididdiga/metadata, da haɓaka ƙarin ingantattun hanyoyin siyasa da lissafi don amfani da albarkatu cikin adalci a cikin ƙungiyar.

9. Nassoshi

1. Ƙungiyar PUNCH4NFDI. (2024). Takardar Fari ta PUNCH4NFDI. [Takardar Ƙungiya ta Ciki].
2. Thain, D., Tannenbaum, T., & Livny, M. (2005). Kwamfuta mai rarrabawa a aikace: gogewar Condor. Haɗin kai da lissafi: gogewa da gogewa, 17(2-4), 323-356.
3. Blomer, J., et al. (2011). Tsarin fayil na CernVM. Journal of Physics: Taron, 331(5), 052004.
4. Takaddun COBalD/TARDIS. (n.d.). An samo daga https://tardis.readthedocs.io/
5. Haɗin gwiwar dCache. (n.d.). dCache: Tsarin ajiya mai rarrabawa. https://www.dcache.org/
6. Haɗin gwiwar XRootD. (n.d.). XRootD: Ingantaccen aiki, samun damar bayanai mai iya jurewa kuskure. http://xrootd.org/
7. Wilkinson, M. D., et al. (2016). Ƙa'idodin Jagorar FAIR don sarrafa bayanan kimiyya da kula da su. Bayanai na kimiyya, 3(1), 1-9.
8. European Open Science Cloud (EOSC). (n.d.). https://eosc-portal.eu/
9. Worldwide LHC Computing Grid (WLCG). (n.d.). https://wlcg.web.cern.ch/