===== Remote media =====

Zde je přehled informací a skriptů vhodných pro zprovoznění funkce remote-media ke zrychlení přihlašování a snížení zátěže serveru.

Diskuze k tomuto proběhla na IRC [[http://irc.minetest.ru/minetest/2014-01-31#i_3567502|#minetest]].

<code text>
[14:36] <Pitriss> https://paste.debian.net/79473/ //tohle by se mohlo hodit když by sme někdy chtěli oddělené media:)
[14:38] <Pitriss> ale pro naše použití je to overkill.. ..byť by to uspíšilo logování do hry
[15:29] <ritchie> Pitriss: otazka je jestli ty oddeleny media muzou snizit load pri logovani
[15:30] <Pitriss> ritchie: můžou, protože to jede threaded.. ale err má to smysl pro 5 lidí?:D
[15:30] <Pitriss> ..vymýšlet jak často to updatovat, rozjíždět to v lightym atd..
[15:30] <ritchie> jj pro 5 lidi to moc smysl nema
[15:37] <Pitriss> tak nějak.. joo kdyby ti to běželo na nějaké hyper VPS a veřejně.. tak to pak asi má smysl.. ale my se tam odpo lognem, večer odlognem..
</code>

== Co je ten mht soubor? ==
  [13:45] <Megaf> whats that mht thing anyway?
  [13:45] <VanessaE> idk, a hashed list of files or something, I don't know exactly how it works

==== Skript od sfan5 ====
[[https://gist.github.com/sfan5/6351560]]

Collects media files from mods/games and puts them in a media directory, also creates an index.mth file in the MTHS format
<file bash collectstatic.sh>
#!/bin/sh
#Options
MINETESTDIR=.
GAMENAME=minetest_game
WORLDDIR=none
MEDIADIR=./media
 
function die {
echo "$1"
exit 1
}
 
if [ ! -d $MINETESTDIR ]; then
die "Please specify a valid Minetest directory"
fi
 
which openssl &> /dev/null || die "Install the OpenSSL commandline tool!"
openssl -h 2>&1 | grep sha1 &> /dev/null || die "OpenSSL without sha1 won't work"
which awk &> /dev/null || die "Install (g)awk!"
 
mkdir -p $MEDIADIR
 
if [ ! $GAMENAME == none ]; then
find $MINETESTDIR/games/$GAMENAME/mods -type f -name "*.png" -o -name "*.ogg" -o -name "*.x" | while read f; do
basename "$f"
cp "$f" $MEDIADIR/`cat "$f" | openssl dgst -sha1 | awk '{print $2}'`
done
fi
 
if [ ! $WORLDDIR == none ]; then
find $WORLDDIR/worldmods -type f -name "*.png" -o -name "*.ogg" -o -name "*.x" | while read f; do
basename "$f"
cp "$f" $MEDIADIR/`cat "$f" | openssl dgst -sha1 | awk '{print $2}'`
done
fi
 
if [ -d $MINETESTDIR/mods ]; then
find $MINETESTDIR/mods -type f -name "*.png" -o -name "*.ogg" -o -name "*.x" | while read f; do
basename "$f"
cp "$f" $MEDIADIR/`cat "$f" | openssl dgst -sha1 | awk '{print $2}'`
done
fi
 
echo -n "Creating index.mth..."
echo -en "MTHS\x00\x01" > $MEDIADIR/index.mth
find $MEDIADIR -type f ! -name index.mth | while read f; do
cat "$f" | openssl dgst -binary -sha1 >> $MEDIADIR/index.mth
done
echo "done"
</file>

==== Skript od RAPHAEL ====
[[https://forum.minetest.net/viewtopic.php?pid=69079#p69079]]

Linux bash script to copy all media files to a single folder:
<file bash remote_media.sh>
    #!/bin/bash

    # Edit to your liking
    MODS="/some/path/minetest/mods/minetest"
    DEFMODS="/some/path/minetest/games/minetest_game/mods"
    DEST="/some/path/minetest/media"

    # Uncomment next two lines if you want to clean out media folder first
    #rm -Rf "$DEST"
    #mkdir "$DEST"

    # Copy files over
    find "$MODS" -type f -name '*.png' -exec cp {} "$DEST" \;
    find "$MODS" -type f -name '*.ogg' -exec cp {} "$DEST" \;
    find "$MODS" -type f -name '*.xcf' -exec cp {} "$DEST" \;
    find "$MODS" -type f -name '*.blend' -exec cp {} "$DEST" \;
    find "$MODS" -type f -name '*.x' -exec cp {} "$DEST" \;

    find "$DEFMODS" -type f -name '*.png' -exec cp {} "$DEST" \;
    find "$DEFMODS" -type f -name '*.ogg' -exec cp {} "$DEST" \;
    find "$DEFMODS" -type f -name '*.xcf' -exec cp {} "$DEST" \;
    find "$DEFMODS" -type f -name '*.blend' -exec cp {} "$DEST" \;
    find "$DEFMODS" -type f -name '*.x' -exec cp {} "$DEST" \;
</file>

EDIT: I still think it would be more useful to have a feature as follows:
  * Server owner compresses cache into say tar tar.gz tar.bz2 tar.7z zip file and upload somewhere
  * Have in minetest.conf a line of maybe: media_archive = http:/url/media.zip
  * On user connect if they need the cache, server sends the url to their client
  * Client downloads cache archive and decompresses where needed
  * Client fetches any media not in archive from server then connects

Benefits of above is that there are plenty of places to host single archives with hotlinking for free. Almost none that can/will host hundreds or thousands of tiny images with hotlinking for free.

==== Skript od Megaf ====
[[https://paste.debian.net/79473/]]

<file bash megaf-remote-media.sh>
#!/bin/bash
	
mkdir /home/megaf/Documents/test/media

# Edit to your liking
MODS="/home/megaf/Documents/test/minetest/minetest/mods"
DEFMODS="/home/megaf/Documents/test/minetest/minetest/games/minetest_game/mods"
MEDIADIR="/home/megaf/Documents/test/media"

# Uncomment next two lines if you want to clean out media folder first
#rm -Rf "$MEDIADIR"
#mkdir "$MEDIADIR"

# Copy files over
find "$MODS" -type f -name '*.png' -exec cp {} "$MEDIADIR" \;
find "$MODS" -type f -name '*.ogg' -exec cp {} "$MEDIADIR" \;
find "$MODS" -type f -name '*.xcf' -exec cp {} "$MEDIADIR" \;
find "$MODS" -type f -name '*.blend' -exec cp {} "$MEDIADIR" \;
find "$MODS" -type f -name '*.x' -exec cp {} "$MEDIADIR" \;

find "$DEFMODS" -type f -name '*.png' -exec cp {} "$MEDIADIR" \;
find "$DEFMODS" -type f -name '*.ogg' -exec cp {} "$MEDIADIR" \;
find "$DEFMODS" -type f -name '*.xcf' -exec cp {} "$MEDIADIR" \;
find "$DEFMODS" -type f -name '*.blend' -exec cp {} "$MEDIADIR" \;
find "$DEFMODS" -type f -name '*.x' -exec cp {} "$MEDIADIR" \;

echo -n "Creating index.mth..."
echo -en "MTHS\x00\x01" > $MEDIADIR/index.mth
find $MEDIADIR -type f ! -name index.mth | while read f; do
cat "$f" | openssl dgst -binary -sha1 >> $MEDIADIR/index.mth
</file>

==== Nastavení pro Lighttpd ====
[[http://www.lighttpd.net/2009/2/16/1-4-21-yes-we-can-do-another-release/]]

New setting to disable returning of a 417 if “Expect: 100-continue” header is given:
  server.reject-expect-100-with-417 = “disable”

Lighttpd vracelo nesprávný kód pro neexistující soubory.
<code text>
[13:40] <VanessaE> 07:39:03: INFO[main]: Client: Failed to remote-fetch 1318 files. Requesting them the usual way.
[13:40] <VanessaE> 07:39:03: INFO[CurlFetchThread]: http://megaf.no-ip.info:8080/media/index.mth no
[13:40] <VanessaE> t found (HTTP response code said error) (response code 417)
...
[13:44] <VanessaE> the curl/media fetch code bugs out when the server returns anything other than 404 for a file not found.
[13:45] <VanessaE> well maybe a browser gets a 404, but the minetest client gets a 417 from your webserver.
</code>