distcc propagator

Helper to propagate a distcc configuration on a compilation network/cluster
distcc Bash JCZD

distcc propagator is a helper script to help propagate a configuration on hosts and clients in a distcc-enabled network ; its features are:

It makes use of a single /etc/distcc/clients file that specifies, for each client (whether it will act as a distcc host or not), the number of local cores to use for compilation. It makes two passes on the clients file, first computing the total number of cores available, then propagating the config on all machines.


# small helper script to ease adding participating hosts in a distcc network
# Author: 	JCZD <jczd@engrenage.ch>
# Date:		2021-12-04
# format of /etc/distcc/clients:
# n,hostname_or_ip
# one per line, where n is the numer of local cores to use for compilation
# - fix the (slight) redundancy between /etc/distcc/{hosts,clients}
# - use a specific authorized_hosts? this adds complexity..
# - read the comments below for more


while read line
	# TODO ignore lines starting with '#'
	local_cores=`echo ${line} | cut -f 1 -d ','`
	hostname=`echo ${line} | cut -f 2 -d ','`
	echo "${hostname} is making ${local_cores} cores available"
	let "total_cores=total_cores+local_cores"
done < /etc/distcc/clients

echo "distcc cluster has ${total_cores} cores available total"

let "N=2*total_cores+1"

while read line
	local_cores=`echo ${line} | cut -f 1 -d ','`
	hostname=`echo ${line} | cut -f 2 -d ','`
	if [ $hostname != `hostname` ]
		echo "remote: ${hostname}"
		echo "-j${N} -l${local_cores}" > /tmp/distcc-propagator-makeopts
		# the line below works once,, then script exits.. why?!? the point was to make sure reverse-connection works
		#ssh $hostname ssh `hostname` echo "Hello from $hostname"
		scp ~/.ssh/authorized_keys $hostname:.ssh >/dev/null
		scp -r /etc/hosts /etc/distcc/ $hostname:/etc/ >/dev/null
		scp /tmp/distcc-propagator-makeopts $hostname:/etc/distcc/makeopts </dev/null	# see Note1
		# TODO test network connectivity, ie. with nmap ; port number should be set in /etc/distcc/clients
		#ssh $hostname /usr/bin/distcc-config --get-hosts
		echo "local: ${hostname}"
		echo "-j${N} -l${local_cores}" > /etc/distcc/makeopts	# see Note1
done < /etc/distcc/clients
# Note1:
# this should be used to set MAKEOPTS in /etc/portage/make.conf, but I am trying 
# to avoid too much parsing and replacing.. any suggestion?