“CSLT Central Storage (CCS)”版本间的差异

来自cslt Wiki
跳转至： 导航、 搜索
 
（5位用户的34个中间修订版本未显示）
第1行： 第1行：
−
'''Why central storage?'''
+
=Why central storage=
  
 
As a research institute, we certainly hope the following two things:
 
As a research institute, we certainly hope the following two things:
  
−
1. All data are easily accessible by all the members, particularly for computing purpose.
+
# All data are easily accessible by all the members, particularly for computing purpose.
−
2. All data are safe, at least for those that are important.
+
# All data are safe, at least for those that are important.
  
 
We then start up the central storage system, which is based on the cluster  and can be accessed in a straightforward way on the cluster per se. If you wish to access this storage from windows, you have to install an NFS client or use sftp.  The most important data are protected by overnight backup.  
 
We then start up the central storage system, which is based on the cluster  and can be accessed in a straightforward way on the cluster per se. If you wish to access this storage from windows, you have to install an NFS client or use sftp.  The most important data are protected by overnight backup.  
  
  
−
'''Storage Configuration'''
+
=Storage Configuration=
  
−
The current configuration involves the following storage blocks:
+
{| class="wikitable"
−
 
+
! journal !! conf !! volume !! backup !! nfs node !! usage !! by time & place
−
/nfs/disk/perm:  the entry for all data and toolkits, weekly backup
+
|-
−
/nfs/disk/work:  the entry for home direcotry and group work, overnight backup
+
| /nfs/disk     || - || 1T || NO || wolf01 || data,tools ||
−
 
+
|-
−
/nfs/asrhome :      asr space, NO backup
+
|/nfs/asrhome   || - || 1T || NO   || grid-6 || ||
−
/nfs/sidhome  :      speaker group space, NO backup
+
|-
−
/nfs/nlphome :      NLP group space, NO backup
+
|/nfs/nlphome   || - || 1T || daily    || grid-6 || ||
−
/nfs/corpus    :    databases, linked to /nfs/disk/perm, weekly backup
+
|-
−
/nfs/project   :      project psace, linked to /nfs/disk/work/project, overnight backup
+
|/nfs/finhome(sidhome)    || - || 1T || daily    || grid-6 || ||
−
/nfs/public      :      public usages(demo, web, cvs etc), linked to /nfs/disk/work/public, overnight backup
+
|-
−
 
+
|/nfs/public    || - || 1T || daily    || grid-6 ||service,release ||
−
IMPORTANT: Since asrhome/sidhome/nlphome are not backed up (at least for now), please use them carefully. The important data should be put under /nfs/disk/work/[asr/sid/nlp].
+
|-
−
 
+
|/freeneb/home   || - || 3.6T || daily    || storage02 || ||
−
 
+
|-
−
'''User and group quota'''
+
|/freeneb/release  || - || 3.6T || daily    || storage02 || ||
−
 
+
|-
−
The space on /nfs/disk/work is valuable, so I will set some quota constraints for each user home dir under /nfs/disk/work/users, as well as the group dir /nfs/disk/work/[asr,sid,nlp]. Please use these backup space for the important things and leaving others to the group home dir (asrhome/sidhome/nlphome). You can also use the /nfs/project to obtain overnight backup, but this should be only so if the project is worthy to do so.
+
|backup  || 6T*6 RAID5 || 29T || NO    || wolf06 ||backup of data corpus0 and /freeneb/release||
−
 
+
|-
−
The user allocation is 10GB, while the group space quota is 100GB.
+
|/nfs/corpus0    || 4T*5 RAID5 || 15T || NO    || storage01 ||data||
 +
|-
 +
|/nfs/corpus1    || 4T*5 RAID5 || 15T || NO    || tiger04 ||data||
 +
|-
 +
|/nfs/corpus2    || 6T*3 RAID0 || 18T || NO    || tiger05 ||data||
 +
|-
 +
|/work   ||  2T*2 RAID0 || 4T || NO    || grid-5 ||working area, down right now||
 +
|-
 +
|/work0    || 4T*2 RAID0 || 8T || NO    || grid-5 ||working area||
 +
|-
 +
|/work1    || 2T*2 RAID0 || 4T || NO    || wolf02 ||working area||
 +
|-
 +
|/work2    || 2T*5 RAID0 || 10T || NO    || grid-10 ||working area, down right now||
 +
|-
 +
|/corpus0 、freeneb/release、work1    || 8T*5 RAID5 || 32T || NO  || wolf06  || backup area ||
 +
|-
 +
|/work3    || 4T*5 RAID5 || 15T || NO  || wolf07  || working area ||
 +
|-
 +
|/work4    || 4T*5 RAID5 || 15T || NO  || wolf07  || working area || 2016-04, MY from JD
 +
|-
 +
|/work5    || 6T*5 RAID5 || 22T || NO  || tiger01  || working area ||
 +
|-
 +
|/work6    || 4T*5 RAID5 || 15T || NO  || tiger03  || working area ||
 +
|-
 +
|/work7    || 4T*5 RAID5 || 15T || NO  || wolf08 || working area ||
 +
|-
 +
|/work8    || 4T*5 RAID5 || 15T || NO  || tiger02 || working area ||
 +
|-
 +
|/work9    || 6T*5 RAID5 || 22T || NO  || tiger07 || working area || 2020-11, DWQ
 +
|-
 +
|/work100    || 4T*5 RAID5 || 15T || NO  || dragon01 || working area ||
 +
|-
 +
|/work101    || 4T*5 RAID5 || 15T || NO  || dragon01 || working area ||
 +
|-
 +
|/work102    || 4T*5 RAID5 || 19T || NO  || dragon02 || working area ||
 +
|-
 +
|/work103    || 4T*5 RAID5 || 19T || NO  || dragon02 || working area ||
 +
|-
 +
|/work104    || 8T*7 RAID5 || 45T || NO  || dragon03 || working area ||2021-08, DWQ
 +
|-
 +
|/work105    || 8T*7 RAID5 || 44T || NO  || dragon04 || working area ||2022-01, DWQ
 +
|}

2022年1月18日 (二) 08:40的最后版本

Why central storage

As a research institute, we certainly hope the following two things:

  1. All data are easily accessible by all the members, particularly for computing purpose.
  2. All data are safe, at least for those that are important.

We then start up the central storage system, which is based on the cluster and can be accessed in a straightforward way on the cluster per se. If you wish to access this storage from windows, you have to install an NFS client or use sftp. The most important data are protected by overnight backup.


Storage Configuration

journal conf volume backup nfs node usage by time & place
/nfs/disk - 1T NO wolf01 data,tools
/nfs/asrhome - 1T NO grid-6
/nfs/nlphome - 1T daily grid-6
/nfs/finhome(sidhome) - 1T daily grid-6
/nfs/public - 1T daily grid-6 service,release
/freeneb/home - 3.6T daily storage02
/freeneb/release - 3.6T daily storage02
backup 6T*6 RAID5 29T NO wolf06 backup of data corpus0 and /freeneb/release
/nfs/corpus0 4T*5 RAID5 15T NO storage01 data
/nfs/corpus1 4T*5 RAID5 15T NO tiger04 data
/nfs/corpus2 6T*3 RAID0 18T NO tiger05 data
/work 2T*2 RAID0 4T NO grid-5 working area, down right now
/work0 4T*2 RAID0 8T NO grid-5 working area
/work1 2T*2 RAID0 4T NO wolf02 working area
/work2 2T*5 RAID0 10T NO grid-10 working area, down right now
/corpus0 、freeneb/release、work1 8T*5 RAID5 32T NO wolf06 backup area
/work3 4T*5 RAID5 15T NO wolf07 working area
/work4 4T*5 RAID5 15T NO wolf07 working area 2016-04, MY from JD
/work5 6T*5 RAID5 22T NO tiger01 working area
/work6 4T*5 RAID5 15T NO tiger03 working area
/work7 4T*5 RAID5 15T NO wolf08 working area
/work8 4T*5 RAID5 15T NO tiger02 working area
/work9 6T*5 RAID5 22T NO tiger07 working area 2020-11, DWQ
/work100 4T*5 RAID5 15T NO dragon01 working area
/work101 4T*5 RAID5 15T NO dragon01 working area
/work102 4T*5 RAID5 19T NO dragon02 working area
/work103 4T*5 RAID5 19T NO dragon02 working area
/work104 8T*7 RAID5 45T NO dragon03 working area 2021-08, DWQ
/work105 8T*7 RAID5 44T NO dragon04 working area 2022-01, DWQ
取自“http://index.cslt.org/mediawiki/index.php?title=CSLT_Central_Storage_(CCS)&oldid=37664”