Viewed   76 times

Because MYSQL's "SELECT" selects integers and floats as strings, and I need every response I get (from JS) to be in a correct data model -

  • 1 not "1",
  • 53.2 not "53.2",

I created this recursive function that works on mixed type - array/object:

private function cast_number(&$mixed) {
    if(is_array($mixed)) {
        foreach ($mixed as $key => $val)
            if (is_numeric($val))
                $mixed[$key] = (double)$val;
            else if (is_array($val) || is_object($val))
                $mixed[$key] = $this->cast_number($val);
    } else if(is_object($mixed)) {
        foreach ($mixed as $key => $val)
            if (is_numeric($val))
                $mixed->$key = (double)$val;
            else if (is_array($val) || is_object($val))
                $mixed->$key = $this->cast_number($val);
    }
    return $mixed;
}

Pretty simple function - if is number, cast double, if is array or object, go over recursivly.

Everything here is in place.

I have two problems with this: - On 6MB of data, of mostly numbers that are represented as strings it took 0.5 seconds - On 200MB of data (yes, I need it, please don't focus on it), it failed after a couple of minutes (usually seconds), saying it needs more than 4GB of memory..

  1. How can I improve this function? (Speed, Memory)
  2. Why is it taking so long? even json_encode, that I would think is a bigger function takes much less time..

 Answers

4

Because coercion used to be faster than casting, I ran this code to calculate timings on PHP 7 :

function getTime($start) {
    return round((microtime(true) - $start) * 1000000) / 1000;
}

function mockData($length) {
    $data = [];
    $i = -1;

    while ($i++ < $length) {
        $data[$i] = strval(rand(1, 10000) / 100);
    }

    return $data;
}

$data = mockData(100000);

// Let's check that they are string before
echo gettype($data[0]) . '<br><br>';

$start = microtime(true);
$convertedData = [];
foreach ($data as $key => $value) {
    $convertedData[$key] = (double) $value;
}
echo '(double) cast took ' . getTime($start) . ' ms.<br>';

$start = microtime(true);
$convertedData = [];
foreach ($data as $key => $value) {
    $convertedData[$key] = 0 + $value;
}
echo 'Coercion took ' . getTime($start) . ' ms.<br>';

And my results are :

(double) cast took 27.508 ms.
Coercion took 28.789 ms.

CONCLUSION

Since using floatval (a third way to achieve string to double conversion) is even longer, you can't do better than that with PHP. What you're trying to achieve is a scripting operation, it shouldn't be used as a normal back-end operation for a web application.

But if you still want to do that, you can higher your memory_limit inside your php.ini file, as long as you don't use the -1 workaround.

UPDATE

I forgot one possible optimization, you should pass your variable by reference to at least execute immediate assignation :

$start = microtime(true);
foreach ($data as $key => $value) {
    $data[$key] = (double) $value;
}
echo getTime($start) . ' ms.<br>';

=> 34.018 ms.

$start = microtime(true);
foreach ($data as &$value) {
    $value = (double) $value;
}
echo getTime($start) . ' ms.<br>';

=> 17.081 ms.

And apparently using coercion with by reference gives even better results :

$start = microtime(true);
foreach ($data as &$value) {
    $value = 0 + $value;
}
echo getTime($start) . ' ms.<br>';

=> 13.1 ms.

Sunday, December 11, 2022
 
5

Neither of them really matter in the big scope of things. The network latency in communicating with the database will far outweigh either the count($object_ids) overhead or the = vs IN overhead. I would call this a case of premature optimization.

You should profile and load-test your application to learn where the real bottlenecks are.

Wednesday, September 28, 2022
3

Here is a nice description of your question: Doing calculations in MySQL vs PHP

In case of the second example the speed issue can be significant. First of all you do not know how big are your comments, so in case of

$x = mysql_query("SELECT * FROM comments");

 while( $res = mysql_fetch_assoc( $x ) ){
   $min_comment = substr( $x['comment'],0,10 ) ;
 }

you ask your server to return you everything (here I mean the whole length of the comment) and this can be significant. Multiplying by the number of rows in the table it can be quite big size of data, which you have to transfer between php and sql. In the second case this SELECT * , SUBSTR(comment, 0, 10) as min_comment FROM comments this will be already done on the server and will not require additional memory.

In case of the first example, I think it is also better to do it on sql side, because you will still need to do additional loop afterwards. Apart from this, people who will be reading your code might be confused why exactly do you need that code.

Monday, August 29, 2022
 
2

It is possible to setup a directory to "in-place-edit" a firefox extension. By this the effort between editing and testing of the Firefox-extension can be reduced.

I have found the good explanation on the blog https://blog.mozilla.org/addons/2009/01/28/how-to-develop-a-firefox-extension/

Here I want to give the principal steps necessary to achieve the "in-place-edit"

Step 1: You have to find your profile directory of Firefox. For example in Linux this would often be often something like this: ~/.mozilla/firefox/#%#%.default/

Step 2: Go to this profile directory

Step 3: If you already have any extensions installed (like for example adblock+ or noscript), then inside this profile directory you will find a folder named extensions. If you do not have yet any additional extension installed, it might be easy to simply install any, only to have the **extensions" folder be setup for you.

Step 4: In this extensions folder you can create a new directory (let us name it "myextensions_1"), which shall contain the stuff of your plugin. This stuff would be the ordinary things like the install.rdf, chrome.manifest files and the content,skin,locale subdirectories. In effect all the stuff you would normaly zip up to become the XPI file.

Step 5. Create a file that is equal to the content of the <em:id> tag that you used in your ìnstall.rdf file. So if you used <em:id>myextensionname@author.org</em:id> you need to create a file named myextensionname@author.org. Inside this file you will write the location of the "in-place-edit-extension-folder" we created before. In our example this we would have

  • the file myextensionname@author.org
  • which contains only the text ~/.mozilla/firefox/#%#%.default/extensions/myextensions_1

Of course the text depends on the location of the folder you use for your plugin.

If you did all things correctly - and maybe double-checked with the instructions of the link above - you can restart or "newly start" firefox. The browser will ask you if you want to allow the usage of the plugin myextensionname@author.org, which you can conceed.

Now you can edit in the folder ~/.mozilla/firefox/#%#%.default/extensions/myextensions_1 and need not to worry about zipping-up -> renaming -> installing. You simple restart Firefox and the edits to your extensions code will become available.

This will allow you swifter and faster developing "in-place".

Thursday, September 29, 2022
4

You can Edit a subitem of the listview (in report mode) using a TEdit, a custom message and handling the OnClick event of the ListView.

Try this sample

Const
  USER_EDITLISTVIEW = WM_USER + 666;

type
  TForm1 = class(TForm)
    ListView1: TListView;
    procedure FormCreate(Sender: TObject);
    procedure ListView1Click(Sender: TObject);
  private
    ListViewEditor: TEdit;
    LItem: TListitem;
    procedure UserEditListView( Var Message: TMessage ); message USER_EDITLISTVIEW;
    procedure ListViewEditorExit(Sender: TObject);
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

uses
  CommCtrl;
const
  EDIT_COLUMN = 2; //Index of the column to Edit

procedure TForm1.FormCreate(Sender: TObject);
Var
  I : Integer;
  Item : TListItem;
begin
  for I := 0 to 9 do
  begin
   Item:=ListView1.Items.Add;
   Item.Caption:=Format('%d.%d',[i,1]);
   Item.SubItems.Add(Format('%d.%d',[i,2]));
   Item.SubItems.Add(Format('%d.%d',[i,3]));
  end;

  //create the TEdit and assign the OnExit event
  ListViewEditor:=TEdit.Create(Self);
  ListViewEditor.Parent:=ListView1;
  ListViewEditor.OnExit:=ListViewEditorExit;
  ListViewEditor.Visible:=False;
end;

procedure TForm1.ListView1Click(Sender: TObject);
var
  LPoint: TPoint;
  LVHitTestInfo: TLVHitTestInfo;
begin
  LPoint:= listview1.ScreenToClient(Mouse.CursorPos);
  ZeroMemory( @LVHitTestInfo, SizeOf(LVHitTestInfo));
  LVHitTestInfo.pt := LPoint;
  //Check if the click was made in the column to edit
  If (ListView1.perform( LVM_SUBITEMHITTEST, 0, LPARAM(@LVHitTestInfo))<>-1) and ( LVHitTestInfo.iSubItem = EDIT_COLUMN ) Then
    PostMessage( self.Handle, USER_EDITLISTVIEW, LVHitTestInfo.iItem, 0 )
  else
    ListViewEditor.Visible:=False; //hide the TEdit 
end;

procedure TForm1.ListViewEditorExit(Sender: TObject);
begin
  If Assigned(LItem) Then
  Begin
    //assign the vslue of the TEdit to the Subitem
    LItem.SubItems[ EDIT_COLUMN-1 ] := ListViewEditor.Text;
    LItem := nil;
  End;
end;

procedure TForm1.UserEditListView(var Message: TMessage);
var
  LRect: TRect;
begin
  LRect.Top := EDIT_COLUMN;
  LRect.Left:= LVIR_BOUNDS;
  listview1.Perform( LVM_GETSUBITEMRECT, Message.wparam,  LPARAM(@LRect) );
  MapWindowPoints( listview1.Handle, ListViewEditor.Parent.Handle, LRect, 2 );
  //get the current Item to edit
  LItem := listview1.Items[ Message.wparam ];
  //set the text of the Edit 
  ListViewEditor.Text := LItem.Subitems[ EDIT_COLUMN-1];
  //set the bounds of the TEdit
  ListViewEditor.BoundsRect := LRect; 
  //Show the TEdit
  ListViewEditor.Visible:=True;
end;
Tuesday, October 25, 2022
Only authorized users can answer the search term. Please sign in first, or register a free account.
Not the answer you're looking for? Browse other questions tagged :