Number of occurances of words in a text (along with similar word)

user_1766219 0
Viewed 53

I'm trying to make a function, that will spot how many times, different words occured in a text. The thing is, that I would like bundle together similar words (and nicknames).

I have this array of interesting words (that I manually have defined):

$interesting_words = [
  'test' => [
    'number_of_occurances' => 0,
    'connected_words' => [
        'TEST',
        'TESTER',
        'TESTING'
      ]
    ],
  'foobar' => [
    'number_of_occurances' => 0,
    'connected_words' => [
        'FOO',
        'FOOBAR',
        'BAR'
      ]
    ]
]

Example text.

Lorem ipsum TEST sit amet, consectetur TESTER elit. Sed in turpis dui. Maecenas venenatis FOOBAR facilisis. Quisque dictum, diam consequat mollis TESTING, orci tellus aliquet nisl, BAR molestie FOO augue at est. In TESTING vehicula lectus. Curabitur ac varius ligula. Pellentesque orci urdna.

Desired output.

Number of occurances for 'test': 4
Number of occurances for 'foobar': 3

Are there a smart way of doing this without having 1.000.000 for-loops?

I'm making the function in Laravel, if that's any help.

laravel php
3 Answers

You can use str_word_count && array_count_values, to get all words occurrences and strtolower to make the search case insensitive when performance and only number of occurrences count :

$words=array_count_values(str_word_count(strtolower($str),1));
foreach($interesting_words as $index=>&$details){
    foreach($details['connected_words'] as $key=>$similar){
        $details['number_of_occurances'] += $words[strtolower($similar)];
    }
}           
print_r($interesting_words );

output:

Array
(
    [test] => Array
        (
            [number_of_occurances] => 4
            [connected_words] => Array
                (
                    [0] => TEST
                    [1] => TESTER
                    [2] => TESTING
                )

        )

    [foobar] => Array
        (
            [number_of_occurances] => 3
            [connected_words] => Array
                (
                    [0] => FOO
                    [1] => FOOBAR
                    [2] => BAR
                )

        )

)
user_101270600

I think it can be done by explode and array_count_values and to make it work In below example I removed . and ,

<?php
$interesting_words = [
  'test' => [
    'number_of_occurances' => 0,
    'connected_words' => [
        'TEST',
        'TESTER',
        'TESTING'
      ]
    ],
  'foobar' => [
    'number_of_occurances' => 0,
    'connected_words' => [
        'FOO',
        'FOOBAR',
        'BAR'
      ]
    ]
];
$str = 'Lorem ipsum TEST sit amet, consectetur TESTER elit. Sed in turpis dui. Maecenas venenatis FOOBAR facilisis. Quisque dictum, diam consequat mollis TESTING, orci tellus aliquet nisl, BAR molestie FOO augue at est. In TESTING vehicula lectus. Curabitur ac varius ligula. Pellentesque orci urdna.';
$str = preg_replace('/[\.\,]/i','',$str);
$str = strtolower($str);
$str_arr = explode(" ",$str);
$str_occurance_counts = array_count_values($str_arr);
foreach($interesting_words as $k=>&$v){
  foreach($v['connected_words'] as $c=>$cVal){
    $v['number_of_occurances'] += $str_occurance_counts[strtolower($cVal)];
  }
}
print_r($interesting_words );
?>

Live Demo Server1

Live Demo Server2

user_28156350 
<?php


$interesting_words = [
  'test' => [
    'number_of_occurances' => 0,
    'connected_words' => [
        'TEST',
        'TESTER',
        'TESTING'
      ]
    ],
  'foobar' => [
    'number_of_occurances' => 0,
    'connected_words' => [
        'FOO',
        'FOOBAR',
        'BAR'
      ]
    ]
];

$testCount=$interesting_words['test']['number_of_occurances'];
$foobarCount=$interesting_words['foobar']['number_of_occurances'];

$text="Lorem ipsum TEST sit amet, consectetur TESTER elit. Sed in turpis dui. Maecenas venenatis 
FOOBAR facilisis. Quisque dictum, diam consequat mollis TESTING, orci tellus aliquet nisl, BAR 
molestie FOO augue at est. In TESTING vehicula lectus. Curabitur ac varius ligula. 
Pellentesque orci urdna.";

$arr= explode(" ", $text);
$numberOfWords=count($arr);
for($i=0;$i<$numberOfWords;$i++)
{
    echo "<br/>";

    if(strpos($arr[$i],'TEST') !== false){
        $testCount=$testCount+1;
    }

    elseif(strpos($arr[$i],'TESTER') !== false){          

    $testCount=$testCount+1;
    }
    elseif(strpos($arr[$i],'TESTING') !== false){

    $testCount=$testCount+1;
    } 

   elseif(strpos($arr[$i],'FOO') !== false){

    $foobarCount=$foobarCount+1;
    }  

   elseif(strpos($arr[$i],'FOOBAR') !== false){

    $foobarCount=$foobarCount+1;
    } 

   elseif(strpos($arr[$i],'BAR') !== false){ 

    $foobarCount=$foobarCount+1;
    }   
}
echo "Number of occurances for 'test':".$testCount;
echo "</br>";
echo "Number of occurances for 'foobar':".$foobarCount;
user_102997280
Related