Skip to content

Commit e811075

Browse files
trystanlea-makertrystanlea-maker
committed
Merge branch 'master' of https://github.com/emoncms/backup
Conflicts: emoncms-export.sh
2 parents d901189 + 923d3cd commit e811075

File tree

9 files changed

+138
-94
lines changed

9 files changed

+138
-94
lines changed

‎backup/backup_controller.php‎

Lines changed: 4 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414

1515
function backup_controller()
1616
{
17-
global $route, $session, $path;
17+
global $route, $session, $path, $redis;
1818
$result = false;
1919

2020
$export_flag = "/tmp/emoncms-flag-export";
@@ -34,14 +34,8 @@ function backup_controller()
3434

3535
if ($route->action == 'start') {
3636
$route->format = "text";
37-
$fh = @fopen($export_flag,"w");
38-
if (!$fh) {
39-
$result = "ERROR: Can't write the flag $export_flag.";
40-
} else {
41-
fwrite($fh,"$export_script>$export_logfile");
42-
$result = "Backup flag set";
43-
}
44-
@fclose($fh);
37+
38+
$redis->rpush("service-runner","$export_script $export_flag>$export_logfile");
4539
}
4640

4741
if ($route->action == 'exportlog') {
@@ -87,15 +81,7 @@ function backup_controller()
8781

8882
if ((move_uploaded_file($_FILES['file']['tmp_name'], $target_path)) && ($uploadOk == 1)) {
8983

90-
$fh = @fopen($import_flag,"w");
91-
if (!$fh) {
92-
$result = "ERROR: Can't write the flag $import_flag.";
93-
} else {
94-
fwrite($fh,"$import_script>$import_logfile");
95-
$result = "Backup flag set";
96-
}
97-
@fclose($fh);
98-
84+
$redis->rpush("service-runner","$import_script $import_flag>$import_logfile");
9985
header('Location: '.$path.'backup');
10086
} else {
10187
$result = "Sorry, there was an error uploading the file";

‎backup/backup_view.php‎

Lines changed: 48 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -30,43 +30,58 @@
3030

3131
<br>
3232
<h2>Backup</h2>
33+
<hr>
3334
<table class="table">
3435
<tr>
35-
<td>
36-
<h3>Export</h3>
37-
<p>Export a compressed archive containing the emoncms mysql database, phpfina, phptimeseries data files, emonhub.conf and emoncms.conf. This can be used to migrate data to another emonpi or emonbase. Refresh page to see archive download link once export is complete.</p>
38-
39-
<pre id="export-log-bound"><div id="export-log"></div></pre>
40-
</td>
41-
<td class="buttons"><br>
42-
<button id="emonpi-backup" class="btn btn-info"><?php echo _('Create backup'); ?></button>
43-
<?php
44-
$backup_filename="emoncms-backup-".date("Y-m-d").".tar.gz";
45-
if (file_exists("/home/pi/data/$backup_filename") && !file_exists("/tmp/backuplock"))
46-
{
47-
echo '<br><br><b>Right Click > Download:</b><br><a href="'.$path.'backup/download">'.$backup_filename.'</a>';
48-
}
49-
?>
50-
</td>
36+
<h3>Export</h3>
37+
<p>Export a compressed archive containing:</p>
38+
<ul>
39+
<li>Emoncms MYSQL database</li>
40+
<li>PHPFINA data files</li>
41+
<li>PHPTIMESERIES data files</li>
42+
<li>EmonHub Config</li>
43+
<li>Emoncms Config</li>
44+
</ul>
45+
<p>These files contain all Emoncms data including:</p>
46+
<ul>
47+
<li>Input processes</li>
48+
<li>Feed data</li>
49+
<li>Dashboards</li>
50+
<li>EmonHub Config</li>
51+
</ul>
52+
<p>The compressed archive can be used to migrate data to another emonPi / emonBase.</p>
53+
<button id="emonpi-backup" class="btn btn-info"><?php echo _('Create backup'); ?></button>
54+
<br><br>
55+
<pre id="export-log-bound"><div id="export-log"></div></pre>
56+
<?php
57+
$backup_filename="emoncms-backup-".date("Y-m-d").".tar.gz";
58+
if (file_exists("/home/pi/data/$backup_filename") && !file_exists("/tmp/backuplock"))
59+
{
60+
echo '<br><br><b>Right Click > Download:</b><br><a href="'.$path.'backup/download">'.$backup_filename.'</a>';
61+
}
62+
?>
63+
<br><br>
64+
<p>Once export is complete refresh page to see download link.</p>
65+
66+
<p><i>Note: Export can take a long some time, please be patient.</i></p>
67+
5168
</tr>
52-
69+
<hr>
5370
<tr>
54-
<td>
55-
<h3>Import</h3>
56-
<p>Import an emoncms backup archive containing the emoncms mysql database, phpfina, phptimeseries data files, emonhub.conf and emoncms.conf.</p>
57-
<p>Before import ensure latest version of Emoncms & emonHub.</p>
58-
<p><b>*CAUTION ALL EMONCMS ACCOUNT DATA WILL BE OVERWRITTEN BY THE IMPORTED DATA*</b></p>
59-
<p>Note: If browser upload fails for large backup files <a href="http://github.com/emoncms/backup">follow manual import instructions.</a> </p>
60-
<pre id="import-log-bound"><div id="import-log"></div></pre>
61-
<br>
62-
<p>Refresh page if log window is not updating. After import is complete logout then log back in using the imported account login details.</p>
63-
</td>
64-
<td>
65-
<form action="<?php echo $path; ?>backup/upload" method="post" enctype="multipart/form-data">
66-
<input type="file" name="file" id="file"><br><br>
67-
<input class="btn btn-danger" type="submit" name="submit" value="Import Backup">
68-
</form>
69-
</td>
71+
<h3>Import</h3>
72+
<p>Import an emoncms backup archive.</p>
73+
<span style="color:red;font-weight:bold;">CAUTION ALL EMONCMS ACCOUNT DATA WILL BE OVERWRITTEN BY THE IMPORTED DATA</span><br><br>
74+
<p><i>Note: Before import update to latest version of Emoncms & emonHub.</i></p>
75+
<form action="<?php echo $path; ?>backup/upload" method="post" enctype="multipart/form-data">
76+
<input type="file" name="file" id="file"><br><br>
77+
<input class="btn btn-danger" type="submit" name="submit" value="Import Backup">
78+
</form>
79+
<br><br>
80+
<p><i>Note: If browser upload fails for large backup files <a href="http://github.com/emoncms/backup">follow manual import instructions.</a></i></p>
81+
<pre id="import-log-bound"><div id="import-log"></div></pre>
82+
<br>
83+
<p><i>Refresh page if log window does not update.</i></p>
84+
<p><i>After import is complete logout then login using the new imported account login details.</i></p>
7085
</tr>
7186

7287
</table>

‎backup/module.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"name" : "Backup",
3+
"version" : "1.1.6"
4+
}

‎config.cfg‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,5 @@ emonhub_config_path="/home/pi/data"
2020
# Location of default fresh specimen emonhub config
2121
emonhub_specimen_config="/home/pi/emonhub/conf"
2222

23-
# Location of emoncms.conf (leave blank if backup not required)
24-
emoncms_config_path="/home/pi/data"
25-
2623
# Location of compressed backup data to import (.tar.gz)
27-
backup_source_path="/home/pi/data/uploads"
24+
backup_source_path="/home/pi/data/uploads"

‎emoncms-export.sh‎

Lines changed: 35 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -4,20 +4,22 @@ date=$(date +"%Y-%m-%d")
44

55
echo "=== Emoncms export start ==="
66
date
7+
echo "Backup module version:"
8+
cat /home/pi/backup/backup/module.json | grep version
79
echo "EUID: $EUID"
810
echo "Reading /home/pi/backup/config.cfg...."
911
if [ -f /home/pi/backup/config.cfg ]
1012
then
1113
source /home/pi/backup/config.cfg
1214
echo "Location of mysql database: $mysql_path"
1315
echo "Location of emonhub.conf: $emonhub_config_path"
14-
echo "Location of emoncms.conf: $emoncms_config_path"
1516
echo "Location of Emoncms: $emoncms_location"
1617
echo "Backup destination: $backup_location"
1718
else
1819
echo "ERROR: Backup /home/pi/backup/config.cfg file does not exist"
1920
exit 1
20-
fi
21+
sudo service feedwriter start > /dev/null
22+
fi
2123

2224
#-----------------------------------------------------------------------------------------------
2325
# Remove Old backup files
@@ -32,7 +34,6 @@ then
3234
sudo rm $backup_location/emoncms-backup-$date.tar
3335
fi
3436

35-
3637
#-----------------------------------------------------------------------------------------------
3738
# Check emonPi / emonBase image version
3839
#-----------------------------------------------------------------------------------------------
@@ -45,17 +46,15 @@ then
4546
echo "Image version: $image_version"
4647
fi
4748

48-
# Detect if SD card image verion, used to restore the correct emonhub.conf
49-
if [[ "$image_date" == "emonSD-17Jun2015" ]]
49+
# Very old images (the ones shipped with kickstarter campaign) have "emonpi-28May2015"
50+
if [[ -z $image_version ]] || [[ "$image_date" == "emonSD-17Jun2015" ]]
5051
then
5152
image="old"
5253
else
5354
image="new"
5455
fi
5556
#-----------------------------------------------------------------------------------------------
5657

57-
58-
5958
sudo service feedwriter stop
6059

6160
# Get MYSQL authentication details from settings.php
@@ -65,29 +64,52 @@ if [ -f /home/pi/backup/get_emoncms_mysql_auth.php ]; then
6564
else
6665
echo "Error: cannot read MYSQL authentication details from Emoncms settings.php"
6766
echo "$PWD"
67+
sudo service feedwriter start > /dev/null
6868
exit 1
6969
fi
7070

7171
# MYSQL Dump Emoncms database
72-
if [ -n "$username" ]; then # if username sring is not empty
72+
if [ -n "$username" ]; then # if username string is not empty
7373
mysqldump -u$username -p$password emoncms > $backup_location/emoncms.sql
74+
if [ $? -ne 0 ]; then
75+
echo "Error: failed to export mysql data"
76+
echo "emoncms export failed"
77+
sudo service feedwriter start > /dev/null
78+
exit 1
79+
fi
80+
7481
else
7582
echo "Error: Cannot read MYSQL authentication details from Emoncms settings.php"
83+
sudo service feedwriter start > /dev/null
7684
exit 1
7785
fi
7886

79-
echo "Emoncms MYSQL database dump complete, adding files to archive .."
87+
echo "Emoncms MYSQL database dump complete, adding to archive..."
8088

8189
# Create backup archive and add config files stripping out the path
82-
tar -cf $backup_location/emoncms-backup-$date.tar $backup_location/emoncms.sql $emonhub_config_path/emonhub.conf $emoncms_config_path/emoncms.conf $emoncms_location/settings.php /home/pi/data/node-red/flows_emonpi.json /home/pi/data/node-red/flows_emonpi_cred.json /home/pi/data/node-red/settings.js --transform 's?.*/??g'
90+
tar -cf $backup_location/emoncms-backup-$date.tar $backup_location/emoncms.sql $emonhub_config_path/emonhub.conf $emoncms_location/settings.php --transform 's?.*/??g' 2>&1
91+
if [ $? -ne 0 ]; then
92+
echo "Error: failed to tar config data"
93+
echo "emoncms export failed"
94+
sudo service feedwriter start > /dev/null
95+
exit 1
96+
fi
8397

98+
echo "Adding phpfina feed data to archive..."
8499
# Append database folder to the archive with absolute path
85-
tar --append --verbose --file=$backup_location/emoncms-backup-$date.tar -C $mysql_path phpfina phptimeseries
100+
tar -vr --file=$backup_location/emoncms-backup-$date.tar -C $mysql_path phpfina
101+
echo "Adding phptimeseries feed data to archive..."
102+
tar -vr --file=$backup_location/emoncms-backup-$date.tar -C $mysql_path phptimeseries
86103

87104
# Compress backup
88105
echo "Compressing archive..."
89-
gzip -f $backup_location/emoncms-backup-$date.tar
90-
106+
gzip -fv $backup_location/emoncms-backup-$date.tar 2>&1
107+
if [ $? -ne 0 ]; then
108+
echo "Error: failed to compress tar file"
109+
echo "emoncms export failed"
110+
sudo service feedwriter start > /dev/null
111+
exit 1
112+
fi
91113

92114
sudo service feedwriter start > /dev/null
93115

‎emoncms-import.sh‎

Lines changed: 5 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,15 @@ data_path="/home/pi/data"
55

66
echo "=== Emoncms import start ==="
77
date +"%Y-%m-%d-%T"
8+
echo "Backup module version:"
9+
cat /home/pi/backup/backup/module.json | grep version
810
echo "EUID: $EUID"
911
echo "Reading /home/pi/backup/config.cfg...."
1012
if [ -f /home/pi/backup/config.cfg ]
1113
then
1214
source /home/pi/backup/config.cfg
1315
echo "Location of mysql database: $mysql_path"
1416
echo "Location of emonhub.conf: $emonhub_config_path"
15-
echo "Location of emoncms.conf: $emoncms_config_path"
1617
echo "Location of Emoncms: $emoncms_location"
1718
echo "Backup destination: $backup_location"
1819
echo "Backup source path: $backup_source_path"
@@ -80,7 +81,7 @@ if [ ! -d $backup_location/import ]; then
8081
sudo chown pi $backup_location/import -R
8182
fi
8283
83-
tar xfz $backup_source_path/$backup_filename -C $backup_location/import 2>&1
84+
tar xfzv $backup_source_path/$backup_filename -C $backup_location/import 2>&1
8485
if [ $? -ne 0 ]; then
8586
echo "Error: failed to decompress backup"
8687
echo "$backup_source_path/$backup_filename has not been removed for diagnotics"
@@ -136,12 +137,10 @@ fi
136137
# cleanup
137138
sudo rm $backup_location/import/emoncms.sql
138139
139-
# Save previous config settings as old.emonhub.conf and old.emoncms.conf
140+
# Save previous config settings as old.emonhub.conf
140141
echo "Import emonhub.conf > $emonhub_config_path/old.emohub.conf"
141142
mv $backup_location/import/emonhub.conf $emonhub_config_path/old.emonhub.conf
142-
echo "Import emoncms.conf > $emonhub_config_path/old.emoncms.conf"
143-
mv $backup_location/import/emoncms.conf $emoncms_config_path/old.emoncms.conf
144-
echo "conf files restored as old.*.conf, original files not modified. Please merge manually."
143+
echo "emonhub.conf files restored as old.emonhub.conf, original file not modified. Please merge manually."
145144
146145
147146
# Start with blank emonhub.conf
@@ -156,11 +155,6 @@ else # Newer Feb15+ image use latest emonhub.conf with MQTT node variable top
156155
cp $emonhub_specimen_config/emonpi.default.emonhub.conf $emonhub_config_path/emonhub.conf
157156
fi
158157
159-
# Create blank emoncms.conf and ensure permissions are correct
160-
sudo touch $emoncms_config_path/emoncms.conf
161-
sudo chown pi:www-data $emoncms_config_path/emoncms.conf
162-
sudo chmod 664 $emoncms_config_path/emoncms.conf
163-
164158
redis-cli "flushall" 2>&1
165159
166160
if [ -f /home/pi/emonpi/emoncmsdbupdate.php ]; then

‎get_emoncms_feed_datadir.php‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
<?php
2+
chdir("/var/www/emoncms");
3+
define('EMONCMS_EXEC', 1);
4+
require "process_settings.php";
5+
6+
foreach ($feed_settings as $engine=>$entry) {
7+
if (isset($feed_settings[$engine]) && isset($feed_settings[$engine]["datadir"]))
8+
echo $engine.'_location="'.$feed_settings[$engine]["datadir"].'"'."\n\n";
9+
}
10+

‎readme.md‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
21
## Emoncms backup export and import tool for backup and migration
32

43
* Export a compressed archive containing Emoncms Inputs, Feed data, Dashboards & config.
@@ -48,7 +47,10 @@ Follow on screen prompts, RasPi will shutdown when process is compleate. It can
4847
git clone https://github.com/emoncms/backup.git
4948
ln -s /home/pi/backup/backup/ /var/www/emoncms/Modules/backup
5049

51-
**Note: Ensure you are running the latest version of Emoncms on the Stable branch. [A change was merged on the 9th Feb 16 to Emoncms core](https://github.com/emoncms/emoncms/commit/e83ad78e6155275d7537104367b8d44ef63d78fe) that enables symlinked modules which is essential for backup module to appear in Emoncms**
50+
**Note:
51+
52+
- Ensure you are running the latest version of Emoncms on the Stable branch. [A change was merged on the 9th Feb 16 to Emoncms core](https://github.com/emoncms/emoncms/commit/e83ad78e6155275d7537104367b8d44ef63d78fe) that enables symlinked modules which is essential for backup module to appear in Emoncms**
53+
- As of June 18 Backup module requires Redis to set service runner flags
5254

5355
**If your running the older 'low-write' branch of Emoncms emonSD-17Jun15 or before then you won't be able to update to the latest version to enable symlinks, to get around this after installing the module browse to [http://emonpi/emoncms/backup](http://emonpi/emoncms/backup)**
5456

@@ -72,16 +74,26 @@ or saving log in var log
7274

7375
## PHP Config
7476

75-
In order to enable uploads of backup zip files we need to set the maximum upload size to be larger than the file we want to upload. This can be set system wide in `/etc/php5/apache2/php.ini`:
77+
In order to enable uploads of backup zip files we need to set the maximum upload size to be larger than the file we want to upload.
78+
79+
If running php5
80+
81+
If using php5
7682

7783
sudo nano /etc/php5/apache2/php.ini
84+
85+
If runnning php7 (Stretch onwards)
86+
7887

88+
sudo nano /etc/php/7.0/apache2/php.ini
89+
7990
Use `[CTRL + W]` to search test
8091

8192
Set:
8293

83-
post_max_size = 200M
84-
upload_max_filesize = 200M
94+
post_max_size = 3G
95+
upload_max_filesize = 3G
96+
upload_tmp_dir = /home/pi/data/uploads
8597

8698
# Create uploads folder
8799

0 commit comments

Comments
 (0)