Tsarin Kwamfuta da Ajiya na Haɗin Kai don PUNCH4NFDI

1. Gabatarwa

Particles, Universe, NuClei and Hadrons for the National Research Data Infrastructure (PUNCH4NFDI) babbar ƙungiya ce ta Jamus wacce DFG (Gidauniyar Bincike ta Jamus) ke tallafawa. Tana wakiltar kimanin masana kimiyya 9,000 daga fagen ilimin barbashi, taurari, barbashi-taurari, hadron, da ilimin nukiliya. Babban manufar ƙungiyar ita ce kafa dandalin bayanan kimiyya na haɗin kai da FAIR (Ana iya gano shi, Samuwa, Haɗin kai, Ana iya sake amfani da shi). Babban ƙalubalen da ake magana da shi shi ne haɗa nau'ikan albarkatun kwamfuta (HPC, HTC, Cloud) da ajiya waɗanda cibiyoyin membobin ke ba da gudummawa "a cikin nau'i" a duk faɗin Jamus, yana ba da damar samun dama ɗaya ga masu bincike.

2. Tsarin Kwamfuta na Haɗin Kai – Compute4PUNCH

An tsara ra'ayin Compute4PUNCH don samar da damar samun dama ga tarin albarkatun lissafi ba tare da sanya canje-canje masu mahimmanci ga tsarin aiki na yanzu a wuraren masu bayarwa ba.

3. Tsarin Ajiya na Haɗin Kai – Storage4PUNCH

Storage4PUNCH yana mai da hankali kan haɗa tsarin ajiya na al'umma, musamman bisa dCache da XRootD, waɗanda suke daidaitattun fasahohi a fagen Kimiyyar Lissafi (HEP). Haɗin kai yana nufin samar da suna ɗaya da tsarin samun dama. Ra'ayin yana kimanta haɗin kai mai zurfi ta hanyar:

• Ka'idojin haɗin ajiya (misali, bisa haɗin kai na mai turawa na XRootD ko mai sarrafa tafki na dCache).
• Matakan ajiya don rage jinkiri da zirga-zirgar WAN.
• Sarrafa metadata don inganta gano bayanai a cikin haɗin kai.

4. Cikakkun Bayanai na Fasaha & Tsarin Lissafi

Ana iya ƙirƙira tsarin tsari na tsakiya a matsayin matsalar ingantawa. Bari $R = \{r_1, r_2, ..., r_n\}$ ya zama tarin albarkatun daban-daban, kowannensu yana da siffofi kamar tsarin gine-gine, ƙwayoyin lissafi masu samuwa $c_i$, ƙwaƙwalwar ajiya $m_i$, da ƙimar farashi/fifiko $p_i$. Aikin $J$ yana da buƙatu $J_{req} = (c_{req}, m_{req}, arch_{req}, t_{req})$. Manufar mai tsara albarkatu ita ce haɓaka amfani gabaɗaya ko ƙarfin aiki.

Ƙimar maki mai sauƙaƙa don sanya aikin $J$ akan albarkatu $r_i$ zai iya zama: $$ S(J, r_i) = \begin{cases} 0 & \text{idan } r_i \text{ bai dace da } J_{req} \\ \alpha \cdot \frac{c_i}{c_{req}} + \beta \cdot \frac{m_i}{m_{req}} - \gamma \cdot p_i & \text{in ba haka ba} \end{cases} $$ inda $\alpha, \beta, \gamma$ suke daidaitattun ma'auni. Tsarin COBalD/TARDIS yana aiwatar da dabaru da madaukai na amsa na ainihi don kusantar irin wannan ingantaccen aiki a hankali, yana daidaitawa da samuwar albarkatu da yanayin jerin ayyuka.

5. Sakamakon Samfuri & Aiki

Bayanin Chati (Ra'ayi): Chati mai layi wanda ke nuna "Ƙarfin Lissafi na Gabaɗaya da ake iya Samuwa akan Lokaci." X-axis shine lokaci (watanni). Ana nuna layi biyu: 1) "Tafkunan Albarkatu ɗaya (An cire haɗin)" – layi masu faɗi, masu tsayawa suna wakiltar ƙarfin tsayayye na wurare daban-daban. 2) "Tafkin Haɗin Kai ta Compute4PUNCH" – layi mafi girma, mai ƙarfi wanda ke ƙaruwa yayin da aka haɗa ƙarin wurare kuma yana nuna ƙananan sauye-sauye, yana nuna daidaita nauyin aiki a cikin haɗin kai. Chatin yana nuna sakamako mai mahimmanci: tsarin haɗin kai yana ba masu amfani babban taron albarkatu mai ƙarfi, mai juriya, kuma ana amfani da shi yadda ya kamata fiye da jimlar sassansa masu keɓancewa.

Samfuran farko sun yi nasara wajen nuna ƙaddamar da aiki daga wurin shiga guda ɗaya (JupyterHub) zuwa tafkunan HTCondor da yawa da tarukan HPC (misali, a KIT, DESY). An aiwatar da ayyukan da ke amfani da yanayin kwantena ta hanyar CVMFS a bayyane akan tsarin gine-gine daban-daban. Ma'auni na farko suna nuna raguwar lokacin jiran aiki ga masu amfani ta hanyar amfani da zagayowar da ba a yi amfani da su ba a cikin haɗin kai, kodayake jinkirin canja wurin bayanai tsakanin wurare ya kasance muhimmin abu ga ayyukan da ke da yawan bayanai.

6. Tsarin Bincike: Nazarin Halin da ake ciki

Halin da ake ciki: Binciken taurari mai yawan saƙo wanda ke danganta bayanai daga na'urar hangen nesa ta neutrino (IceCube) da wurin kallon gamma-ray (CTA).

Aiki ba tare da Haɗin Kai ba: Dole ne mai bincike ya: 1. Nemi rabon lissafi daban-daban akan tarin HPC don siminti da gona HTC don sarrafa abubuwan da suka faru. 2. Canja manyan bayanai (ma'aunin TB) tsakanin tsarin ajiya a cibiyoyi daban-daban da hannu. 3. Sarrafa yanayin software daban-daban da hanyoyin tabbatar da asali.

Aiki tare da Compute4PUNCH/Storage4PUNCH: 1. Mai bincike ya shiga cikin PUNCH JupyterHub tare da alama guda ɗaya. 2. An ayyana tsarin aikin bincike (misali, ta amfani da Snakemake ko makamantansu). Ayyukan siminti (wanda ya dace da HPC) ana tura su ta atomatik ta hanyar TARDIS zuwa albarkatun HPC masu dacewa. Ana aika ayyukan sarrafa abubuwan da suka faru masu yawan aiki zuwa gonakin HTC. 3. Tsarin aikin yana nuna bayanai ta sunan ajiya na haɗin kai (misali, `punch://data/icecube/run_xyz.root`). Haɗin kai na XRootD/dCache yana sarrafa wuri da canja wuri. 4. Duk ayyukan suna jawo yanayin software ɗaya daga CVMFS. Wannan binciken yana nuna yuwuwar canji: mai bincike yana mai da hankali kan kimiyya, ba tsarin abubuwan more rayuwa ba.

7. Ayyukan Gaba & Taswirar Ci Gaba

Abubuwan more rayuwa na PUNCH4NFDI suna shimfiɗa tushe don ayyuka masu ci gaba da yawa:

• Horar da Na'ura ta Koyon Haɗin Kai: Yin amfani da GPUs daban-daban a wurare daban-daban don horar da ƙirar manyan sifofi, mai yuwuwa ta amfani da tsarin kamar PyTorch ko TensorFlow tare da algorithms na koyo na haɗin kai da aka daidaita don bayan HTCondor/TARDIS.
• Sanya Nauyin Aiki mai Ƙarfi, Mai Bin Manufa: Haɗa tsarin tsari mai sanin carbon, inda ake tura ayyuka zuwa wuraren da ke da samuwar makamashi mai sabuntawa, kamar ra'ayoyin da Green Algorithms ke bincika.
• Haɗin Kai tsakanin Ƙungiyoyi: Zama tsari don haɗawa da sauran ƙungiyoyin NFDI ko ƙungiyoyin Turai kamar European Open Science Cloud (EOSC), ƙirƙirar abubuwan more rayuwa na bincike na Turai.
• Ajiya da Farko na Bayanai mai Hikima: Yin amfani da asalin aiki da nazarin hasashe don ajiye bayanai da gangan a wuraren lissafi, rage jinkirin WAN, ƙalubalen da kuma ke tsakiya ga ayyuka kamar IRIS-HEP.

8. Ra'ayin Mai Bincike: Fahimta ta Tsaki, Tsarin Ma'ana, Ƙarfafawa & Kurakurai, Shawarwari masu Amfani

Fahimta ta Tsaki: PUNCH4NFDI ba ta gina sabon babban kwamfuta ba; tana gina sashe na ƙirƙira da tsari wanda ke juya yanayin binciken lissafi na Jamus mai rarrabuwa, mai rarrabuwa zuwa abin amfani mai haɗin kai, mai da hankali kan mai amfani. Wannan dabarar "haɗin kai akan maye gurbin" ce ta gargajiya, tana ba da fifikon karɓuwa da haɓakawa akan canji mai juyi—wani yunkuri mai hikima idan aka yi la'akari da gaskiyar siyasa da aiki na cibiyoyin da aka ba da kuɗaɗen jama'a.

Tsarin Ma'ana: Ma'ana tana da inganci: 1) Amincewa da bambancin kaddarorin da mallaka (albarkatun suna zaune tare da cibiyoyi). 2) Sanya ƙananan sabbin buƙatu (amfani da alamomi, kwantena). 3) Sanya sashe na tsaka-tsaki mai hikima, mai daidaitawa (COBalD/TARDIS) don ƙaddamar da rikitarwa. 4) Samar da sauƙaƙan, masu hulɗa na zamani (JupyterHub). 5) Haɗa bayanai iri ɗaya don kammala madauki. Littafin haɗin kai ne daga ƙasa zuwa sama wanda sauran ƙungiyoyi suka yi bincike.

Ƙarfafawa & Kurakurai: Ƙarfafawa: Amfani da abubuwan da aka gwada (HTCondor, dCache, CVMFS) daga al'ummar HEP yana rage haɗarin fasaha sosai. Mayar da hankali kan AAI da kwantena yana magance manyan abubuwan da ke hana karɓuwa guda biyu: samun dama da software. Zaɓin COBalD/TARDIS yana da wahayi—mai tsara lissafi ne mai sauƙi, na tushen Python wanda aka tsara don wannan yanayin haɗin gwiwar gajimare na dama. Kurakurai Masu Muhimmanci: Giwa a cikin ɗaki shine motsin bayanai. Haɗa lissafi ya fi sauƙi fiye da haɗa ajiya. Takardar ta ambaci ajiya da kimanta metadata, amma matsalolin ingantaccen aikin suna na duniya, farashin canja wurin bayanai na WAN, da aiwatar da manufofin bayanai tsakanin wurare kawai an yi ishara da su. Ba tare da ingantaccen mafita a nan ba, tafkin lissafi na haɗin kai zai kasance cikin matsala ga ayyukan da ke da yawan bayanai. Bugu da ƙari, nasarar ta dogara ne akan ci gaba da ba da gudummawar "a cikin nau'i" daga membobi—tsarin tattalin arziki mai rauni.

Shawarwari masu Amfani: 1. Ga PUNCH4NFDI: Ƙara ƙarfi akan sashe na bayanai. Haɗin gwiwa da ƙarfi tare da ayyuka kamar Rucio don sarrafa bayanai da Open Science Grid don ƙwarewar aiki. Haɓaka SLAs masu bayyanawa tare da masu ba da albarkatu, musamman game da farashin fita na bayanai. 2. Ga Masu Gasa/Masu Koyi: Kar a kwafi tsarin gine-gine kawai. Darasin ainihi yana cikin mulki da ƙirar haɗin kai mai sauƙi. Fara da samfuri mai aiki akan ƴan wuraren da suka yarda kuma ku girma ta hanyar halitta. 3. Ga Masu Sayarwa & Ƙungiyoyin Bayar da Kuɗi: Wannan samfurin yana nuna cewa zuba jari na gaba na lissafin bincike ya kamata ya ba da kuɗi don haɗa tsaka-tsaki da dorewar software (kamar COBalD) gwargwadon, idan ba fiye da, kayan aiki na ɗanyan ba. Ku ba da kuɗin "manne".

A ƙarshe, tsarin PUNCH4NFDI babban darasi ne a cikin injiniyanci na abubuwan more rayuwa na yanar gizo mai hikima. Ya gane cewa babban ƙalubale a cikin lissafin kimiyya ba sau da yawa ba FLOPS bane, amma amfani da samun dama. Idan sun iya warware goro na bayanai na haɗin kai, za su ƙirƙiri samfuri mai yuwuwar gaske don sake fasalin ba kawai Jamus ba, amma Turai, lissafin bincike.

9. Nassoshi

1. Ƙungiyar PUNCH4NFDI. (2024). Takardar Fari ta PUNCH4NFDI. NFDI.
2. Thain, D., Tannenbaum, T., & Livny, M. (2005). Lissafi na rarraba a aikace: gwanin Condor. Haɗin kai da Lissafi: Gwanin Gwaninta, 17(2-4), 323-356.
3. Giffels, M., et al. (2023). COBalD/TARDIS - Rufe albarkatu mai ƙarfi don lissafin dama. Journal of Physics: Taron Taro.
4. Blomer, J., et al. (2011). Tsarin Fayil na CernVM. Journal of Physics: Taron Taro, 331(5), 052004.
5. Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Fassarar Hotuna zuwa Hotuna marasa Haɗin gwiwa ta amfani da Cibiyoyin Adawa na Madauki. Proceedings of the IEEE International Conference on Computer Vision (ICCV). (An ambata a matsayin misali na hanyar lissafi mai canzawa wanda zai iya amfani da irin wannan abubuwan more rayuwa na haɗin kai).
6. Haɗin gwiwar dCache. (2023). dCache: Tsarin ajiya mai rarrabawa. https://www.dcache.org.
7. Haɗin gwiwar XRootD. (2023). XRootD: Babban aiki, samun dama mai ɗorewa ga bayanai. https://xrootd.slac.stanford.edu.
8. European Open Science Cloud (EOSC). (2024). https://eosc-portal.eu.