Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to prevent file cleaning, touch cache, its link, md5 and id, resolves #1282 #1367

Open
wants to merge 10 commits into
base: dev
Choose a base branch
from
Open
37 changes: 36 additions & 1 deletion BrainPortal/app/models/data_provider.rb
Original file line number Diff line number Diff line change
Expand Up @@ -1353,7 +1353,43 @@ def self.cleanup_leftover_cache_files(do_it=false, options={})
end
end

# Updates the time stamp for important auxiliary directories and files
# as workaround for HPC file deletion policies.
#
# Some Bourreaux systems are configured with disk allocations where files older than N days are erased automatically.
# To prevent such system from deleting the top-level directories for the DP_Cache, and some cbrain-specific files,
# the boot process should touch them to reset their timestamps.
#
# For a portal or bourreau:
#
# - the +DataProvider+ cache dir
# - the +DP_Cache_Key.md5+ and
# - +DP_Cache_Rev.id+ located in that cache dir
#
# For a bourreau:
#
# - the +gridshare+ dir
# - the +DP_Cache+ symbolic link located in it.
def self.system_touch
myself = RemoteResource.current_resource
cache_dir = myself.dp_cache_dir
dp_cache_id = File.join cache_dir, DataProvider::DP_CACHE_ID_FILE
dp_cache_md5 = File.join cache_dir, DataProvider::DP_CACHE_MD5_FILE

FileUtils.touch [cache_dir, dp_cache_id, dp_cache_md5], verbose: true, nocreate: true

# touch only cache for Portal, for Bourreau touch gridshare
return true unless myself.is_a? Bourreau

gridshare_dir = myself.cms_shared_dir
sym_path = File.join gridshare_dir, DataProvider::DP_CACHE_SYML

FileUtils.touch gridshare_dir, verbose: true, nocreate: true

# update timestamp for a softlink rather than the folder it points to
# note, --no-dereference works on major os but not all of them
system("touch", "--no-dereference", sym_path) # --no-create is implied, at least for Rocky and Ubuntu
end

#################################################################
# Access restriction checking methods, using flags in meta-data.
Expand Down Expand Up @@ -1615,4 +1651,3 @@ def self.local_rsync_protects_args?
end

end

18 changes: 18 additions & 0 deletions BrainPortal/lib/cbrain_system_checks.rb
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,24 @@ def self.a050_check_data_provider_cache_wipe #:nodoc:
end
end

# prevents archiving/delete of cbrain system files and top directories, related to
# cache and gridshare
def self.a060_ensure_system_files_will_not_be_deleted #:nodoc:

#-----------------------------------------------------------------------------
puts "C> Updating timestamp for important system files and directories"
#-----------------------------------------------------------------------------

cache_root = DataProvider.cache_rootdir rescue nil
# Need to perform a `to_s` due to a strange behaviour of `blank?`
# on `Pathname` (if a content of a `Pathname` is empty it will return true)
if cache_root.to_s.blank?
puts "C> \t- SKIPPING! No cache root directory yet configured!"
return
end

DataProvider.system_touch
end


def self.a080_ensure_set_starttime_revision #:nodoc:
Expand Down
Loading