Remote media
Zde je přehled informací a skriptů vhodných pro zprovoznění funkce remote-media ke zrychlení přihlašování a snížení zátěže serveru.
Diskuze k tomuto proběhla na IRC #minetest.
[14:36] <Pitriss> https://paste.debian.net/79473/ //tohle by se mohlo hodit když by sme někdy chtěli oddělené media:) [14:38] <Pitriss> ale pro naše použití je to overkill.. ..byť by to uspíšilo logování do hry [15:29] <ritchie> Pitriss: otazka je jestli ty oddeleny media muzou snizit load pri logovani [15:30] <Pitriss> ritchie: můžou, protože to jede threaded.. ale err má to smysl pro 5 lidí?:D [15:30] <Pitriss> ..vymýšlet jak často to updatovat, rozjíždět to v lightym atd.. [15:30] <ritchie> jj pro 5 lidi to moc smysl nema [15:37] <Pitriss> tak nějak.. joo kdyby ti to běželo na nějaké hyper VPS a veřejně.. tak to pak asi má smysl.. ale my se tam odpo lognem, večer odlognem..
Co je ten mht soubor?
[13:45] <Megaf> whats that mht thing anyway? [13:45] <VanessaE> idk, a hashed list of files or something, I don't know exactly how it works
Skript od sfan5
https://gist.github.com/sfan5/6351560
Collects media files from mods/games and puts them in a media directory, also creates an index.mth file in the MTHS format
- collectstatic.sh
#!/bin/sh #Options MINETESTDIR=. GAMENAME=minetest_game WORLDDIR=none MEDIADIR=./media function die { echo "$1" exit 1 } if [ ! -d $MINETESTDIR ]; then die "Please specify a valid Minetest directory" fi which openssl &> /dev/null || die "Install the OpenSSL commandline tool!" openssl -h 2>&1 | grep sha1 &> /dev/null || die "OpenSSL without sha1 won't work" which awk &> /dev/null || die "Install (g)awk!" mkdir -p $MEDIADIR if [ ! $GAMENAME == none ]; then find $MINETESTDIR/games/$GAMENAME/mods -type f -name "*.png" -o -name "*.ogg" -o -name "*.x" | while read f; do basename "$f" cp "$f" $MEDIADIR/`cat "$f" | openssl dgst -sha1 | awk '{print $2}'` done fi if [ ! $WORLDDIR == none ]; then find $WORLDDIR/worldmods -type f -name "*.png" -o -name "*.ogg" -o -name "*.x" | while read f; do basename "$f" cp "$f" $MEDIADIR/`cat "$f" | openssl dgst -sha1 | awk '{print $2}'` done fi if [ -d $MINETESTDIR/mods ]; then find $MINETESTDIR/mods -type f -name "*.png" -o -name "*.ogg" -o -name "*.x" | while read f; do basename "$f" cp "$f" $MEDIADIR/`cat "$f" | openssl dgst -sha1 | awk '{print $2}'` done fi echo -n "Creating index.mth..." echo -en "MTHS\x00\x01" > $MEDIADIR/index.mth find $MEDIADIR -type f ! -name index.mth | while read f; do cat "$f" | openssl dgst -binary -sha1 >> $MEDIADIR/index.mth done echo "done"
Skript od RAPHAEL
https://forum.minetest.net/viewtopic.php?pid=69079#p69079
Linux bash script to copy all media files to a single folder:
- remote_media.sh
#!/bin/bash # Edit to your liking MODS="/some/path/minetest/mods/minetest" DEFMODS="/some/path/minetest/games/minetest_game/mods" DEST="/some/path/minetest/media" # Uncomment next two lines if you want to clean out media folder first #rm -Rf "$DEST" #mkdir "$DEST" # Copy files over find "$MODS" -type f -name '*.png' -exec cp {} "$DEST" \; find "$MODS" -type f -name '*.ogg' -exec cp {} "$DEST" \; find "$MODS" -type f -name '*.xcf' -exec cp {} "$DEST" \; find "$MODS" -type f -name '*.blend' -exec cp {} "$DEST" \; find "$MODS" -type f -name '*.x' -exec cp {} "$DEST" \; find "$DEFMODS" -type f -name '*.png' -exec cp {} "$DEST" \; find "$DEFMODS" -type f -name '*.ogg' -exec cp {} "$DEST" \; find "$DEFMODS" -type f -name '*.xcf' -exec cp {} "$DEST" \; find "$DEFMODS" -type f -name '*.blend' -exec cp {} "$DEST" \; find "$DEFMODS" -type f -name '*.x' -exec cp {} "$DEST" \;
EDIT: I still think it would be more useful to have a feature as follows:
- Server owner compresses cache into say tar tar.gz tar.bz2 tar.7z zip file and upload somewhere
- Have in minetest.conf a line of maybe: media_archive = http:/url/media.zip
- On user connect if they need the cache, server sends the url to their client
- Client downloads cache archive and decompresses where needed
- Client fetches any media not in archive from server then connects
Benefits of above is that there are plenty of places to host single archives with hotlinking for free. Almost none that can/will host hundreds or thousands of tiny images with hotlinking for free.
Skript od Megaf
https://paste.debian.net/79473/
- megaf-remote-media.sh
#!/bin/bash mkdir /home/megaf/Documents/test/media # Edit to your liking MODS="/home/megaf/Documents/test/minetest/minetest/mods" DEFMODS="/home/megaf/Documents/test/minetest/minetest/games/minetest_game/mods" MEDIADIR="/home/megaf/Documents/test/media" # Uncomment next two lines if you want to clean out media folder first #rm -Rf "$MEDIADIR" #mkdir "$MEDIADIR" # Copy files over find "$MODS" -type f -name '*.png' -exec cp {} "$MEDIADIR" \; find "$MODS" -type f -name '*.ogg' -exec cp {} "$MEDIADIR" \; find "$MODS" -type f -name '*.xcf' -exec cp {} "$MEDIADIR" \; find "$MODS" -type f -name '*.blend' -exec cp {} "$MEDIADIR" \; find "$MODS" -type f -name '*.x' -exec cp {} "$MEDIADIR" \; find "$DEFMODS" -type f -name '*.png' -exec cp {} "$MEDIADIR" \; find "$DEFMODS" -type f -name '*.ogg' -exec cp {} "$MEDIADIR" \; find "$DEFMODS" -type f -name '*.xcf' -exec cp {} "$MEDIADIR" \; find "$DEFMODS" -type f -name '*.blend' -exec cp {} "$MEDIADIR" \; find "$DEFMODS" -type f -name '*.x' -exec cp {} "$MEDIADIR" \; echo -n "Creating index.mth..." echo -en "MTHS\x00\x01" > $MEDIADIR/index.mth find $MEDIADIR -type f ! -name index.mth | while read f; do cat "$f" | openssl dgst -binary -sha1 >> $MEDIADIR/index.mth
Nastavení pro Lighttpd
http://www.lighttpd.net/2009/2/16/1-4-21-yes-we-can-do-another-release/
New setting to disable returning of a 417 if “Expect: 100-continue” header is given:
server.reject-expect-100-with-417 = “disable”
Lighttpd vracelo nesprávný kód pro neexistující soubory.
[13:40] <VanessaE> 07:39:03: INFO[main]: Client: Failed to remote-fetch 1318 files. Requesting them the usual way. [13:40] <VanessaE> 07:39:03: INFO[CurlFetchThread]: http://megaf.no-ip.info:8080/media/index.mth no [13:40] <VanessaE> t found (HTTP response code said error) (response code 417) ... [13:44] <VanessaE> the curl/media fetch code bugs out when the server returns anything other than 404 for a file not found. [13:45] <VanessaE> well maybe a browser gets a 404, but the minetest client gets a 417 from your webserver.