I was wondering if it is possible to speed up the existing import process I created for a wordpress site? I need to import the data from external xml file, create products based on that data, and dowmload the images and assign them to created products. The problem is that thumbnails also need to be generated and associated to the products via wp functions. I currently manage to insert 1 to 2 images per sec, the script is called every 5 min using cron job and limit for import is 120 images. That equals to approx 80 to 150 sec for a block of 120 images. Tre problem is that i need to import around 10000 products with 200000 images.
in tmp folder per time is approx 1000 images
Current functionality that i use is:
function upload_image(){
ini_set("memory_limit","2048M");
add_filter('intermediate_image_sizes', function ($image_sizes){
return array('thumbnail');
},1000 );
if($this->ids===false) {
self::get_all_ids();
}
$upload_dir = wp_upload_dir();
$root = $upload_dir['basedir'].DIRECTORY_SEPARATOR.'tmp'.DIRECTORY_SEPARATOR;
$handle = opendir($root);
$files=array();
$propertys=array();
$propertys_images=array();
while (false !== ($entry = readdir($handle))) {
if ($entry != "." && $entry != "..") {
$tmp_data=explode('_',$entry);
if(isset($tmp_data[1])) {
$post_id = isset($this->ids[$tmp_data[1]]) ? $this->ids[$tmp_data[1]] : 0;
if (!empty($post_id)) {
$propertys[$post_id] = $post_id;
$files[] = array('url' => $root . $entry, 'post_id' => $post_id, 'name' => $entry);
}else{
unlink( $root . $entry);
}
}else{
unlink( $root . $entry);
}
}
}
if (empty($propertys)){return'';}
global $wpdb;
$results = $wpdb->get_results("
SELECT ID,post_parent, post_title
FROM $wpdb->posts AS posts
WHERE post_type = 'attachment'
AND post_parent IN(".implode(',',$propertys).")
Order by ID asc
",ARRAY_A ) ;
$set_images=array();
foreach ($results as $r){
$set_images[]=$r['post_title'];
$propertys_images[$r['post_parent']][$r['ID']]=$r['ID'];
}
foreach ($files as $k => $d){
if(in_array($d['name'],$set_images)){
unset($files[$k]);
unlink($d['url']);
}
}
require_once( ABSPATH . 'wp-admin/includes/image.php' );
foreach ($files as $id_of_i => $f){
if($id_of_i > 120){continue;}
$image_url = $f['name'];
$post_id=$f['post_id'];
$base_path=$upload_dir['basedir'].DIRECTORY_SEPARATOR.'tmp' .DIRECTORY_SEPARATOR.$image_url;
$filename = basename( $image_url );
if ( wp_mkdir_p( $upload_dir['path'] ) ) {
$file = $upload_dir['path'] . '/' . $filename;
}
else {
$file = $upload_dir['basedir'] . '/' . $filename;
}
if(file_exists($base_path)) {
rename($base_path, $file);
//copy($base_path, $file);
}
$wp_filetype = wp_check_filetype( $filename, null );
$attachment = array(
'post_mime_type' => $wp_filetype['type'],
'post_title' => sanitize_file_name( $filename ),
'post_parent' => $post_id,
'post_content' => '',
'post_status' => 'inherit'
);
$attach_id = wp_insert_attachment( $attachment, $file );
$propertys_images[$post_id][$attach_id]=$attach_id;
$attach_data = wp_generate_attachment_metadata( $attach_id, $file );
wp_update_attachment_metadata( $attach_id, $attach_data );
}
foreach ($propertys_images as $prop_id => $images){
$images_ids=array_values($images);
set_post_thumbnail($prop_id, $images_ids[0]);
update_post_meta($prop_id, '_thumbnail_id', $images_ids[0]);
update_post_meta($prop_id, 'fave_prop_slider_image', $images_ids[0]);
$num=get_post_meta($prop_id,'numb_of_images',true);
delete_post_meta($prop_id,'fave_property_images');
foreach ($images_ids as $k =>$id){
if($k!='0'){
add_post_meta($prop_id, 'fave_property_images', $id);
}
}
if($num<=count($images_ids)){
update_post_meta($prop_id, 'all_images_imp', 'true');
}
}
exit();
}
First i set to only generate thumbnail of this images, then i collect all product_codes( "get_all_ids()" ) then i get all images from tmp folder and check if product for that img exist, then get all attachment for products whose images i have then i migrate image to wp upload path
then call "wp_insert_attachment" to add base image, then using "wp_generate_attachment_metadata" i generate a thumbnail and then that information i append to attachment using "wp_update_attachment_metadata"
received attachment ids i add to product and product gallery's
i think that in this script the slowed part is generating a thumbnail is there any possibility to speed up this process, or if you have any suggestion
with current speed it will took approx 5 - to 10 days to add all the images
Thank you in advance