Filesystem Scripts

batch_renamer

Renames a directory of files based on a template.

First we’ll examine the contents of a directory and ensure it only contains files with names we want to rename.

>>> os.listdir("/path/to/dir")
['img_1074.jpg', 'img_1076.jpg', 'img_1077.jpg']

As we can see it does we’ll then invoke the script like so.

python3 batch_renamer.py /path/to/dir
img_1074.jpg --> Ashley_0.jpg
img_1076.jpg --> Ashley_1.jpg
img_1077.jpg --> Ashley_2.jpg

class pyutil.batch_renamer.BatchRename(template)

Bases: string.Template

Delimiter for string substitutions.

delimiter = '%'

pattern = re.compile('\n %(?:\n (?P<escaped>%) | # Escape sequence of two delimiters\n (?P<named>(?a:[_a-z][_a-z0-9]*)) | # delimiter and a Python identifier\n {(?P<braced>(?a:[_a-z][_a-z0-9]*)), re.IGNORECASE|re.VERBOSE)

pyutil.batch_renamer._parse_arguments()

Parse user arguments.

pyutil.batch_renamer.batch_mover(pattern, new_pattern, directory=None)

Move files in the current working directory that match a pattern.

Parameters

• pattern (str) – Pattern to check filenames for. Functionally would be the old extension.

• new_pattern (str) – What to replace the old pattern with.

• directory (str, optional) – Directory where files that need to be renamed can be found.

Returns

Return type

Bool

pyutil.batch_renamer.file_check(pattern, file_to_check)

Check that the file exists and matched the desired pattern.

pyutil.batch_renamer.main()

Execute module.

dir_cleaner

Deletes extraneous files.

Without frequent monitoring, directories like /tmp and /var/log can frequently grow to sizes that are difficult to manage because of clutter and files.

However, there has to be a middle ground between deleting thousands of files one by one and rm -rf /tmp/.

This module attempts that.

Initially tested on the android app Termux, this specifically deletes directories with only month old sockets.

• $PREFIX/tmp/nvim

• $PREFIX/tmp/ssh

Note

On Ubuntu the big one is /var/log/journal so we might need to remind the user for credentials. getpass.getpass()?

In addition, it felt like a good way to get more familiar with the new pathlib module.

Concrete Roadmap

Up until now this has been fairly general and quite abstract.

Let’s come up with a few pre-planned and clear endpoints.

One way to implement this module would be:

1. Compare packages that are installed to what’s contained /usr/share/doc. I have like 4500 folders in that directory and nowhere near that many packages installed. Cruft and accumulation frequently doesn’t get cleared out of there correctly.

pyutil.dir_cleaner.clean(ftype='*.pyc', recursive=False)

Remove all pyc files. Add input for filetype later.

Parameters

• ftype (filetype) – File to iterately remove.

• recursive (Bool, Optional) – Whether to search the directory recursively or not.

pyutil.dir_cleaner.dir_cleaner(i)

Yield all directories before november.

pyutil.dir_cleaner.main()

Directory cleaner.

pyutil.dir_cleaner.rmtree(path, ignore_errors=False, onerror=None)

Returns shutil.rmtree().

dlink

Symlink all of the files in one directory into another.

Synopsis

This module can be used to create individual symlinks to every file in a directory. This is a huge convenience wen symlinking dotfiles or configuration files held in a different location than where the software of interest expects it.

This is quite easily one of my most frequently used scripts.

Usage

ln -s path/to/dest/* [path/to/src]

As an example, one can git clone dotfiles in a directory named projects or src. The location of the git repository is irrelevant, and as such, we’ll refer to it as src from here, as dest as it’s where the symlinks point to.

For example if wants symlinks pointing to /home/User/dotfiles/.vim, then running dlink.py in /home/User/.vim with $HOME/dotfiles/.vim as an argument will create symlinks in $HOME/.vim pointing to $HOME/dotfiles/.vim.

Inspiration

If the module is given 2 args, the intended response is for it to behave similarly to the classic Unix idiom:

ln -s path/to/dest/* [path/to/src]

or treat it similarly to os.symlink().

pyutil.dlink.dlink(dest, src)

Symlinks a directory from another one.

Parameters

• dest (str) – The directory where the original files are located.

• src (str) – Optional argument indicating the directory where the symlinks are to be created. If the src argument isn’t provided, it is assumed that the current working directory is the src dir.

dlink2

This is a rewrite of a script I’ve had for years, so I decided to go above and beyond.

The idea behind it was to create something that would easily allow for creating arbitrarily nested trees of symlinks.

Todo

Sep 17, 2019: Recursive option

Currently doesnt work.

Todo

literally what

$: dlink2.py  ~/projects/viconf/.config/nvim/rplugin/python3
Traceback (most recent call last):
File "/data/data/com.termux/files/home/bin/dlink2.py", line 233, in <module>
main()
File "/data/data/com.termux/files/home/bin/dlink2.py", line 210, in main
args = user_arguments.parse_args()
AttributeError: 'NoneType' object has no attribute 'parse_args'

See also

IPython.utils.path.ensure_dir_exists()function

Check for a dir and create it if it doesn’t exist.

exception pyutil.dlink2.PermissionsError

Bases: OSError

Symlinking error typically from Windows.

pyutil.dlink2.dlink(destination_dir, source_dir, is_recursive=False, glob_pattern=None)

Symlink user provided files.

The module doesn’t immediately check for correct permissions or operating system.

As a result, the onus is put on the user to ensure that the necessary requirements per OS are met.

Namely on Windows 10, that if symlinks are allowed on the filesystem, whether they can only be created by an administrator. Recent enough versions of Windows 10 have introduced the ability for regular users to create symlinks as well as admins.

Parameters

• destination_dir (str) – Directory where symlinks point to.

• source_dir (str, optional) – Directory where symlinks are created.

• recursive (bool, optional) – Whether to recursively symlink directories beneath the destination_dir. Defaults to False.

• glob_pattern (str) – Only symlink files that match a certain pattern.

pyutil.dlink2.generate_dest(dest, glob_pattern=None)

Return a generator for all the files in the destination directory.

pyutil.dlink2.get_basenames(directory)

Get the basenames of all the files in a directory.

pyutil.dlink2.main()

Set up the module.

pyutil.dlink2.symlink(src, dest)

Execute the symlinking part of this.

linktree

From the package “python3.6-examples” in the Ubuntu repositories.

Minor modifications for flake8, pydocstyle etc.

Mar 10, 2019:

Added logging.

Make a copy of a directory tree with symbolic links to all files in the original tree.

All symbolic links go to a special symbolic link at the top, so you can easily fix things if the original source tree moves.

This would probably get a huge improvement in readability from pathlib.

See also

mkreal

pyutil.linktree._check_existence(directory)

Check that a directory exists.

pyutil.linktree._parse_arguments()

Handle user inputs.

pyutil.linktree.linknames(old, new, link)

Recursively symlink a directory tree.

pyutil.linktree.main()

Handle user inputs and initialize arguments.

mv_to_repo

Move files from the home directory to the dotfiles repo.

This is a script I’ve been using for the better part of a year, so while the docstring formatting isn’t consistent and there are a couple odd sections, this script has served a very utilitiarian purpose.

May refactor one day. But it continues to work.

Note

This module assumes a python interpreter above version 3.4.

pyutil.mv_to_repo.backup_file(src)

Backs up file src. Utilizes shutil.copy2().

Parameters

src (str) – File to backup

pyutil.mv_to_repo.main()

Dispatch the remaining implementation of the module.

Determine if a file name is in the current directory or absolute path.

Then set up a relative path from $HOME. Use the root of the repo as the new root and move the file there, all while creating directories and backups.

Runs checks, calls func to backup file src, moves it to the dotfiles repo and symlinks it.

Moves file to a hardcoded path but will be generalized to take as an argument.

Assumes

User runs the script from inside the folder of the file they want to move.

pyutil.mv_to_repo.repo_dir_check(dest)

Check that the directory is in the repository and make one otherwise.

Useful info about mkdir:

To mimic behavior of mkdir -p, use flags parents=True and exists_ok=True

Parameters

dest (str) – Checks that the file to move has a corresponding directory in the repo