“CSLT Central Storage (CCS)”版本间的差异

来自cslt Wiki
跳转至： 导航、 搜索
第1行： 第1行：
−
'''Why central storage?'''
+
=Why central storage=
  
 
As a research institute, we certainly hope the following two things:
 
As a research institute, we certainly hope the following two things:
  
−
1. All data are easily accessible by all the members, particularly for computing purpose.
+
# All data are easily accessible by all the members, particularly for computing purpose.
−
2. All data are safe, at least for those that are important.
+
# All data are safe, at least for those that are important.
  
 
We then start up the central storage system, which is based on the cluster  and can be accessed in a straightforward way on the cluster per se. If you wish to access this storage from windows, you have to install an NFS client or use sftp.  The most important data are protected by overnight backup.  
 
We then start up the central storage system, which is based on the cluster  and can be accessed in a straightforward way on the cluster per se. If you wish to access this storage from windows, you have to install an NFS client or use sftp.  The most important data are protected by overnight backup.  
  
  
−
'''Storage Configuration'''
+
=Storage Configuration=
  
−
The current configuration involves the following storage blocks:
+
==Storage space==
  
 
'''/nfs/disk/perm''':  the entry for all data and toolkits, weekly backup
 
'''/nfs/disk/perm''':  the entry for all data and toolkits, weekly backup
第23行： 第23行：
 
'''/nfs/nlphome''' :      NLP group space, NO backup
 
'''/nfs/nlphome''' :      NLP group space, NO backup
  
−
'''/nfs/corpus'''    :    databases, linked to /nfs/disk/perm, weekly backup
+
'''/nfs/corpus'''    :    databases, linked to /nfs/disk/perm, NO backup
  
−
'''/nfs/project'''    :     project space, linked to /nfs/disk/work/project, overnight backup
+
'''/nfs/corpus0'''    :     RAID5, 12TB, 1-to-1 backup of /nfs/corpus
  
−
'''/nfs/public'''     :      public usage(demo, web, cvs etc), linked to /nfs/disk/work/public, overnight backup
+
'''/nfs/project'''   :      project space, NO backup
  
−
IMPORTANT: Since asrhome/sidhome/nlphome are not backed up (at least for now), please use them carefully. The important data should be put under /nfs/disk/work/[asr/sid/nlp].
+
'''/nfs/public'''      :      public usage(demo, web, cvs etc), overnight backup
  
  
−
'''User and group quota'''
+
==Working space==
 +
'''/work''': 4TB 2*2disk RAID0
  
−
The space on /nfs/disk/work is valuable, so I will set some quota constraints for each user home dir under /nfs/disk/work/users, as well as the group dir /nfs/disk/work/[asr,sid,nlp]. Please use these backup space for the important things and leaving others to the group home dir (asrhome/sidhome/nlphome). You can also use the /nfs/project to obtain overnight backup, but this should be only so if the project is worthy to do so.
+
'''/work0''': 12TB 3*4 disk RAID0
−
 
+
−
The user allocation is '''10GB''', while the group space quota is '''200GB'''. Let me know if you have special requests.
+
−
 
+
−
'''Restore your files'''
+
−
 
+
−
In any case that you want to recove your files (either by disk failure or unwilling deleting), just logon grid-1 and then copy from /mirrordisk
+
−
 
+
−
'''For admin'''
+
−
 
+
−
1. The  backup crond scripts are in on grid-0 /mirrordisk/utl, the crontab is in /etc/cron.d
+
−
2. The  quota uses quota-tools.
+

2014年4月15日 (二) 14:13的版本

Why central storage

As a research institute, we certainly hope the following two things:

  1. All data are easily accessible by all the members, particularly for computing purpose.
  2. All data are safe, at least for those that are important.

We then start up the central storage system, which is based on the cluster and can be accessed in a straightforward way on the cluster per se. If you wish to access this storage from windows, you have to install an NFS client or use sftp. The most important data are protected by overnight backup.


Storage Configuration

Storage space

/nfs/disk/perm: the entry for all data and toolkits, weekly backup

/nfs/disk/work: the entry for home direcotry and group work, overnight backup

/nfs/asrhome  : asr space, NO backup

/nfs/sidhome  : speaker group space, NO backup

/nfs/nlphome : NLP group space, NO backup

/nfs/corpus  : databases, linked to /nfs/disk/perm, NO backup

/nfs/corpus0  : RAID5, 12TB, 1-to-1 backup of /nfs/corpus

/nfs/project  : project space, NO backup

/nfs/public  : public usage(demo, web, cvs etc), overnight backup


Working space

/work: 4TB 2*2disk RAID0

/work0: 12TB 3*4 disk RAID0

取自“http://index.cslt.org/mediawiki/index.php?title=CSLT_Central_Storage_(CCS)&oldid=9712”