CSV file to update MySQL DB

I have a file that my company had created to pull data from three CSV files, delte the CSV files, delete the contents of the DB, and add the new data into the DB.

Here is the script:

[php]

<?php //ini_set("display_errors", '1'); //error_reporting(E_ALL & ~E_NOTICE); // Determine if the three files exist, if not exit if(file_exists("loads.csv") && file_exists("loads-OB.csv") && file_exists("loads-IB.csv")) { // cycle through fuser until three files aren't being used by other program (like ftp) $rc1 = 0; $rc2 = 0; $rc3 = 0; $fuser_count = 0; while (($rc1== 0) || ($rc2 == 0) || ($rc3 == 0)) { // include a counter. If number of cycles > n, send email and exit $fuser_count++; if($fuser_count > 180) { // pick a number // mail about the problem and quit $file_error = "can't access csv file - open for three minutes"; mail_clint($file_error); exit; } sleep(1); // wait one second $last_line = exec("fuser loads.csv", $output, $rc1); $last_line = exec("fuser loads-OB.csv", $output, $rc2); $last_line = exec("fuser loads-IB.csv", $output, $rc3); } // Read the contents of each file into an array $loads = file("loads.csv"); $loads_ob = file("loads-OB.csv"); $loads_ib = file("loads-IB.csv"); // Delete the files unlink('loads.csv'); unlink('loads-OB.csv'); unlink('loads-IB.csv'); /*// Create empty output string $bigfile = ''; */ // connect to database include('../other/define.php'); $dbobj = new mysqli(DBHOST, DBUSER, PASSW, USEDB); if ($dbobj->connect_errno) { $db_error = "Can't connect to mysql or database: " . $dbobj->connect_error; echo $db_error; mail_clint($db_error); exit(); } // delete contents of loads table $query = "TRUNCATE TABLE loads"; if(!$dbobj->query($query)) { $db_error = "couldn't empty loads table: " . $dbobj->error; echo $db_error; mail_clint($db_error); exit(); } // set big array to contain the three arrays $big_array = array($loads, $loads_ob, $loads_ib); // for each array in big array foreach ($big_array as $csv_file) { // for each array as line foreach($csv_file as $line) { // each field in the line is enclosed in quotes! // convert string to an array using ", (quote + comma) as separator $linearray = explode('",', rtrim($line)); // get rid of the rest of the quotes foreach($linearray as &$field) { $field = str_replace('"', '', $field); } // get first seven fields $firstpart = array_slice($linearray, 0 , 7); // secondpart = array_slice(line, -5) get the last five $secondpart = array_slice($linearray, -5); // Create the human readable timestamp date_default_timezone_set('America/Chicago'); $now = date('m/d/y - H:i'); // insert the combined array into the database $nextline = array_merge($firstpart, $secondpart); $query = "INSERT INTO loads (load_num, stops, avail_date, pickup_city, pickup_state, deliv_city, deliv_state, description, equipment, broker, broker_phone, broker_email, updated) VALUES ('$nextline[0]', '$nextline[1]', '$nextline[2]', '$nextline[3]', '$nextline[4]', '$nextline[5]', '$nextline[6]', '$nextline[7]', '$nextline[8]', '$nextline[9]', '$nextline[10]', '$nextline[11]', '$now')"; if(!$dbobj->query($query)) { $db_error = "couldn't insert data into table: " . $dbobj->error; echo $db_error; mail_clint($db_error); exit(); } /*// combine and turn back into a string with newline $nextline = implode(',', array_merge($firstpart, $secondpart)) . "\n"; // add to output string $bigfile .= $nextline; */ } } /*// output string now has contents of all three files // output the big string to the temp file file_put_contents("temp_data.csv", $bigfile); chmod("temp_data.csv", 0644); */ /*// use LOAD DATA INFILE to put contents of file into loads table $query = "LOAD DATA INFILE './temp_data.csv' INTO TABLE loads FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (load_num, stops, avail_date, pickup_city, pickup_state, deliv_city, deliv_state, description,equipment,broker,broker_phone,broker_email)"; if(!$dbobj->query($query)) { $db_error = "couldn't load new data into table: " . $dbobj->error; echo $db_error; //mail_clint($db_error); exit(); } */ echo "database updated"; } function mail_clint($msg) { require_once('../other/class.phpmailer.php'); $mail = new PHPMailer; $mail->AddAddress('[email protected]); $mail->Subject = "Problem Updating Database"; $mail->Body = $msg; $nail->Send(); } ?>

[/php]

What I would like to do is modify this script for another project so that it imports the data from a single CSV file, deletes the CSV file from the server, and then updates the “products” table with new inventory qty where “products_model” in the table equals column A from the CSV file and columnB=“products_quantity”. I do not want the info deleted from the server, but instead updated. I also need it to check and and ignore any data from column A from the CSV data that are not in the DB as there are some products in the CSV file that I do not have listed on the website.

Would anyone be willing to assist me in editing this script?

Thanks,

Clint

I am on my way out today but I will try and look at this when I get home or tomorrow. Sorry I cannot do it now.

Any chance yo uhave had a chance to look at this yet?

Thanks,

Clint

Sorry I completely spaced it… I’ll do it now, first thing I noticed is your missing a ’ in the $mail = AddAddress('[email protected]); line. Should be

$mail = AddAddress(‘[email protected]’);

Second your post ends with $nail->Send(); Should be $mail->Send();

I never caught the $nail so thanks. the .com’ was me screwing up when I was removing url for email.

Here is what I have been able to do so far which is not much but if will give you the proper file name, remember, for this one, I am only wanting to grab 1 csv file, but I need it capable of adding more.

[php]

<?php //ini_set("display_errors", '1'); //error_reporting(E_ALL & ~E_NOTICE); // Determine if the three files exist, if not exit if(file_exists("IM-QTY-CSV.csv")) { // cycle through fuser until all files aren't being used by other program (like ftp) $rc1 = 0; $fuser_count = 0; while (($rc1== 0)) { // include a counter. If number of cycles > n, send email and exit $fuser_count++; if($fuser_count > 180) { // pick a number // mail about the problem and quit $file_error = "can't access csv file - open for three minutes"; mail_clint($file_error); exit; } sleep(1); // wait one second $last_line = exec("fuser IM-QTY-CSV.csv", $output, $rc1); } // Read the contents of each file into an array $rsr_inventory = file("IM-QTY-CSV.csv"); echo $rsr_inventory; // Delete the files unlink('IM-QTY-CSV.csv'); /*// Create empty output string $bigfile = ''; */ // connect to database include('define.php'); $dbobj = new mysqli(DB_HOST, DB_USER, DB_PASS, DB_NAME); if ($dbobj->connect_errno) { $db_error = "Can't connect to mysql or database: " . $dbobj->connect_error; echo $db_error; mail_clint($db_error); exit(); } // set big array to contain the three arrays $big_array = array($rsr_inventory); // for each array in big array foreach ($big_array as $csv_file) { // for each array as line foreach($csv_file as $line) { // each field in the line is enclosed in quotes! // convert string to an array using ", (quote + comma) as separator $linearray = explode('",', rtrim($line)); // get rid of the rest of the quotes foreach($linearray as &$field) { $field = str_replace('"', '', $field); } // get first seven fields $firstpart = array_slice($linearray, 0 , 7); // secondpart = array_slice(line, -5) get the last five $secondpart = array_slice($linearray, -5); // Create the human readable timestamp date_default_timezone_set('America/Chicago'); $now = date('m/d/y - H:i'); // insert the combined array into the database $nextline = array_merge($firstpart, $secondpart); $query = "UPDATE products (load_num, stops, avail_date, pickup_city, pickup_state, deliv_city, deliv_state, description, equipment, broker, broker_phone, broker_email, updated) VALUES ('$nextline[0]', '$nextline[1]', '$nextline[2]', '$nextline[3]', '$nextline[4]', '$nextline[5]', '$nextline[6]', '$nextline[7]', '$nextline[8]', '$nextline[9]', '$nextline[10]', '$nextline[11]', '$now')"; if(!$dbobj->query($query)) { $db_error = "couldn't update data into table: " . $dbobj->error; echo $db_error; mail_clint($db_error); exit(); } /*// combine and turn back into a string with newline $nextline = implode(',', array_merge($firstpart, $secondpart)) . "\n"; // add to output string $bigfile .= $nextline; */ } } /*// output string now has contents of all three files // output the big string to the temp file file_put_contents("temp_data.csv", $bigfile); chmod("temp_data.csv", 0644); */ /*// use LOAD DATA INFILE to put contents of file into loads table $query = "LOAD DATA INFILE './temp_data.csv' INTO TABLE loads FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (load_num, stops, avail_date, pickup_city, pickup_state, deliv_city, deliv_state, description,equipment,broker,broker_phone,broker_email)"; if(!$dbobj->query($query)) { $db_error = "couldn't load new data into table: " . $dbobj->error; echo $db_error; //mail_clint($db_error); exit(); } */ echo "database updated"; } function mail_clint($msg) { require_once('class.phpmailer.php'); $mail = new PHPMailer; $mail->AddAddress('[email protected]'); $mail->Subject = "Problem Updating Database"; $mail->Body = $msg; $mail->Send(); } ?>

[/php]

Not sure if this helps or anything.

THanks though for your assistance!!!

Clint

you should really fix your spacing you do not need a blank line in between every line of code… makes it very difficult to read. I am working on it now give me a sec and I will have something. If you have a demo CSV file I can test that would be great.

The spacing was not mine. This was a file that my company paid another developer to do.

You can download the csv file from http://tacticaloffense.com/forum/IM-QTY-CSV.csv since I can’t see how to upload docs here.

Thanks again for your help!

Sorry again work got in the way… I will be back to this ASAP (tonight)

:smiley:

I am glad there are people like you in the world! I appreciate everything that you are doing!

Any luck?

[php]$file = fopen(‘IM-QTY-CSV.csv’, ‘r’);
while (($line = fgetcsv($file)) !== FALSE) {
//$line is an array of the csv elements
$productnumber = $line[0];
$inventorynumber = $line[1];
// you will need to change table to your table name and product id and inventroy to your table rows.
$sql = “UPDATE table SET product id = '”.$productnumber."’, inventory = ‘".$inventorynumber."’ WHERE table.product id = ‘$productnumber’";
$res = mysqli_query($mysqli, $sql) or die($mysqli)
echo ‘the following information was entered into the database’;
echo $productnumber.’ ‘.$inventorynumber.’
’;

}
mysql_close($mysqli);
fclose($file);[/php]

here is the mysql query. I have not tested in inserting into a database and I did not put it into your code but you should be able to get it to work. If you need more help let me know.

Here it is in your code… I am not sure if it will work I have not tested it but if it does not work let me know and I will work out the bugs on my dev server.

[php]<?php
//ini_set(“display_errors”, ‘1’);
//error_reporting(E_ALL & ~E_NOTICE);
// Determine if the three files exist, if not exit
if(file_exists(“IM-QTY-CSV.csv”)) {

// cycle through fuser until all files aren't being used by other program (like ftp)
$rc1 = 0;
$fuser_count = 0;
while (($rc1== 0)) {
	// include a counter. If number of cycles > n, send email and exit
	$fuser_count++; 
	if($fuser_count > 180) {  // pick a number
		// mail about the problem and quit
		$file_error = "can't access csv file  - open for three minutes";
		mail_clint($file_error);
		exit;
	}
	
	sleep(1);  // wait one second
	$last_line = exec("fuser IM-QTY-CSV.csv", $output, $rc1);
}

// Read the contents of each file into an array
$rsr_inventory = file("IM-QTY-CSV.csv");

echo $rsr_inventory;

// Delete the files
unlink('IM-QTY-CSV.csv');

/*// Create empty output string
$bigfile = ''; */

// connect to database
include('define.php');
$mysqli = new mysqli(DB_HOST, DB_USER, DB_PASS, DB_NAME);
if ($mysqli->connect_errno) {
	$db_error = "Can't connect to mysql or database: " . $mysqli->connect_error;
	echo $db_error;
	mail_clint($db_error);
	exit();
}


$file = fopen('IM-QTY-CSV.csv', 'r');
while (($line = fgetcsv($file)) !== FALSE) {
	//$line is an array of the csv elements
	$productnumber = $line[0];
	$inventorynumber = $line[1];
	// you will need to change table to your table name and product id and inventroy to your table rows.
	$sql = "UPDATE table SET `product id` = '".$productnumber."', `inventory` = '".$inventorynumber."' WHERE table.`product id` = '$productnumber'";
	$res = mysqli_query($mysqli, $sql) or die($mysqli)
	echo 'the following information was entered into the database';
	echo $productnumber.' '.$inventorynumber.'<br>';

}

echo "database updated";

}

function mail_clint($msg) {
require_once(‘class.phpmailer.php’);
$mail = new PHPMailer;
$mail->AddAddress(‘[email protected]’);
$mail->Subject = “Problem Updating Database”;
$mail->Body = $msg;
$mail->Send();
}
mysql_close($mysqli);
fclose($file);
?>
[/php]

I cut out all the fuser stuff in this version I do not think you need it and it was throwing timeout errors at me. here is the code without it.

[php]<?php
ini_set(“display_errors”, ‘1’);
error_reporting(E_ALL & ~E_NOTICE);
// Determine if the three files exist, if not exit
if(file_exists(“IM-QTY-CSV.csv”)) {

// connect to database
include('define.php');
$mysqli = new mysqli(DB_HOST, DB_USER, DB_PASS, DB_NAME);
if ($mysqli->connect_errno) {
	$db_error = "Can't connect to mysql or database: " . $mysqli->connect_error;
	echo $db_error;
	mail_clint($db_error);
	exit();
}

echo 'The following information was entered into the database <br/>';
$file = fopen('IM-QTY-CSV.csv', 'r');
while (($line = fgetcsv($file)) !== FALSE) {
	//$line is an array of the csv elements
	$productnumber = $line[0];
	$inventorynumber = $line[1];
	// you will need to change table to your table name and product id and inventroy to your table rows.
	$sql = "UPDATE table SET `product id` = '".$productnumber."', `inventory` = '".$inventorynumber."' WHERE table.`product id` = '$productnumber'";
	$res = mysqli_query($mysqli, $sql) or die($mysqli)
	echo $productnumber.' '.$inventorynumber.'<br>';
}
echo "database updated";

}

function mail_clint($msg) {
require_once(‘class.phpmailer.php’);
$mail = new PHPMailer;
$mail->AddAddress(‘[email protected]’);
$mail->Subject = “Problem Updating Database”;
$mail->Body = $msg;
$mail->Send();
}

mysql_close($mysqli);
fclose($file);
?>
[/php]

You’re awesme!

I added a line of code to delete the csv file once updated marked with the color markeup “marron” (it didnt actually change the color, line 32 and 33). I edited the update script to fit my table. There is also an error though at the bottom of the page:

Warning: mysql_close(): supplied argument is not a valid MySQL-Link resource in /home/content/36/6701936/html/_db_update/update_db.php on line 43

which is
[php]mysql_close($mysqli);[/php]

I tried swapping it with the
[php]fclose($file);[/php]

but still got the same result.

here is the script with the added line:

[php]

<?php ini_set("display_errors", '1'); error_reporting(E_ALL & ~E_NOTICE); // Determine if the three files exist, if not exit if(file_exists("IM-QTY-CSV.csv")) { // connect to database include('define.php'); $mysqli = new mysqli(DB_HOST, DB_USER, DB_PASS, DB_NAME); if ($mysqli->connect_errno) { $db_error = "Can't connect to mysql or database: " . $mysqli->connect_error; echo $db_error; mail_clint($db_error); exit(); } echo 'The following information was entered into the database
'; $file = fopen('IM-QTY-CSV.csv', 'r'); while (($line = fgetcsv($file)) !== FALSE) { //$line is an array of the csv elements $productnumber = $line[0]; $inventorynumber = $line[1]; // you will need to change table to your table name and product id and inventroy to your table rows. $sql = "UPDATE products_test SET `products_quantity` = '".$inventorynumber."' WHERE `products_model` = '$productnumber'"; $res = mysqli_query($mysqli, $sql) or die($mysqli); echo $productnumber.' '.$inventorynumber.'
'; } echo "database updated"; // Delete the files unlink('IM-QTY-CSV.csv'); } function mail_clint($msg) { require_once('class.phpmailer.php'); $mail = new PHPMailer; $mail->AddAddress('[email protected]'); $mail->Subject = "Problem Updating Database"; $mail->Body = $msg; $mail->Send(); } mysql_close($mysqli); fclose($file); ?>

[/php]

Do you know why we am getting that error?

Again, thank you for all of your work! It does update the table as it is supposed to.

The mysql_close line just closes the connection to the database you can comment it out or remove it if you want should not affect the code and stop throwing the error.

The color issue is that those color statements are not valid php or any other markup language that I know of. If you are trying to get the script to tell you that it deleted the file at the end of the message in maroon then you need something like

[php]echo “<span sytle=“color:maroon;”>The file $file has been deleted.”;[/php]

I was just putting the color in there to show you the added code. Thank you for all your help! ;D ;D ;D ;D ;D ;D ;D ;D ;D ;D ;D

All good. If you would not mind can you mark the topic solved (very bottom) and plus my karma (under my name).

Sponsor our Newsletter | Privacy Policy | Terms of Service