Problem with share_cloned on Perl dbh objects

huangapple go评论53阅读模式
英文:

Problem with share_cloned on Perl dbh objects

问题

我试图测试 Perl 的 ConnectionPool 概念。基本上,测试是使用共享数组来保存预先存在的 $dbh 实例,以便各个线程可以重复使用它们,而不必每次都启动连接以浪费时间/开销。问题是测试在非线程模式下运行正常,但在线程模式下出现错误。似乎 shared_clone() 方法无法成功创建 Pool 的共享副本?请参阅下面的测试代码。任何见解都不胜感激!

英文:

I'm trying to test the ConnectionPool concept for Perl. Basically the test is to have a shared array that holds pre-existing $dbh instances so that individual threads can just re-use them without wasting time/overhead of starting a connection every time. Problem is the test runs fine in non-thread mode but errors out in thread mode. It seems like the shared_clone() method does not successfully create shared copies of the Pool? Please see the test code below. Any insight is greatly appreciated!

use strict;
use warnings 'all';
no warnings 'uninitialized';
use threads;
use threads::shared;
use Thread::Semaphore; 
use DBI ;
sub say { print @_, "\n" }

my $testDB = 'TestDB.db' ;
if ( -e $testDB ) { unlink( $testDB ); }
#Set up DB and create table with some data
my $dbName = "dbi:SQLite:dbname=$testDB" ;
my $userId = '' ;
my $password = '' ;
my $dbh = DBI->connect( $dbName, $userId, $password ) ;
$dbh->do( "create table Tbl1 ( id integer, name char(35) )" ) ;
$dbh->do( "insert into Tbl1 (id, name) values (1, 'Connection')" ) ;
$dbh->do( "insert into Tbl1 (id, name) values (2, 'Pool')" ) ;

#Testing ConnectionPool concept
my $connPool = [ $dbh, $dbh->clone(), $dbh->clone() ] ;
my $connPoolShared = shared_clone( $connPool ) ;

#Test non-threading env.
say "========= Test non-threading first ========" ;
for ( my $i=0; $i < scalar( @$connPool ); $i++ ) {
  say "Is non-threading dbh member $i pingable? " . $connPool->[$i]->ping() ;
}#end for

#Test threading env.
say "\n========= Test threading next ========" ;
my @threadList ;
my $semaphore = Thread::Semaphore->new() ;
for ( my $count = 1; $count <= 3; $count++ ) {
  my $t = threads->new( \&sub1 ) ;
  push( @threadList, $t ) ;
}#end for

foreach ( @threadList ) {
  $_->join() ; 
}#end for

sub sub1 {
  $semaphore->down() ;
    my $threadDbh = pop( @$connPoolShared ) ;
    say "Is threading dbh pingable? " . $threadDbh->ping() ;   
  $semaphore->up() ;
}#end sub

say "End Test" ;

答案1

得分: 1

Nice idea. However, the Threads and Thread Safety section in DBI says:

> If the DBI and drivers are loaded and handles created before the thread is created then it will get a cloned copy of the DBI, the drivers and the handles.
>
> However, the internal pointer data within the handles will refer to the DBI and drivers in the original interpreter. Using those handles in the new interpreter thread is not safe, so the DBI detects this and croaks on any method call using handles that don't belong to the current thread (except for DESTROY).
>
> Because of this (possibly temporary) restriction, newly created threads must make their own connections to the database. Handles can't be shared across threads.

So it seems that you are out of luck, right off the bat (because of the way threading works).

I did find the take_imp_data method, which:

> Leaves the $dbh in an almost dead, zombie-like, state and returns a binary string of raw implementation data from the driver which describes the current database connection. Effectively it detaches the underlying database API connection data from the DBI handle.

This, they say, can be useful as:

> Why would you want to do this? You don't, forget I even mentioned it. Unless, that is, you're implementing something advanced like a multi-threaded connection pool.
...
The returned $imp_data can be passed as a dbi_imp_data attribute to a later connect() call, even in a separate thread in the same process, where the driver can use it to 'adopt' the existing connection that the implementation data was taken from.

This appears to open a door, but then it goes:

> Some things to keep in mind...
...
> * using the same $imp_data to create more than one other new $dbh at a time may well lead to unpleasant problems. Don't do that.

Then I'm not sure how one would create that multi-threaded pool, since taking implementation data once makes the connection a zombie, while that "soul" can't be used more than once.

Since the phrase "may well lead to unpleasant problems" is a vague and unspecified "threat" — warning of possible subtle problems in the future — I didn't try it out as I can't ascertain whether there are problems and of what kind.

Hopefully creating new connections won't be prohibitively expensive for your needs and purposes so that you can make your pool that way.

英文:

Nice idea. However, the Threads and Thread Safety section in DBI says

> If the DBI and drivers are loaded and handles created before the thread is created then it will get a cloned copy of the DBI, the drivers and the handles.
>
> However, the internal pointer data within the handles will refer to the DBI and drivers in the original interpreter. Using those handles in the new interpreter thread is not safe, so the DBI detects this and croaks on any method call using handles that don't belong to the current thread (except for DESTROY).
>
> Because of this (possibly temporary) restriction, newly created threads must make their own connections to the database. Handles can't be shared across threads.

So it seems that you are out of luck, right off the bat (because of the way threading works).


I did find the take_imp_data method, which

> Leaves the $dbh in an almost dead, zombie-like, state and returns a binary string of raw implementation data from the driver which describes the current database connection. Effectively it detaches the underlying database API connection data from the DBI handle.

This, they say, can be useful as

> Why would you want to do this? You don't, forget I even mentioned it. Unless, that is, you're implementing something advanced like a multi-threaded connection pool.
...
The returned $imp_data can be passed as a dbi_imp_data attribute to a later connect() call, even in a separate thread in the same process, where the driver can use it to 'adopt' the existing connection that the implementation data was taken from.

This appears to open a door, but then it goes

> Some things to keep in mind...
...
> * using the same $imp_data to create more than one other new $dbh at a time may well lead to unpleasant problems. Don't do that.

Then I'm not sure how one would create that multi-threaded pool, since taking implementation data once makes the connection a zombie, while that "soul" can't be used more than once.

Since the phrase "may well lead to unpleasant problems" is a vague and unspecified "threat" — warning of possible subtle problems in the future — I didn't try it out as I can't ascertain whether there are problems and of what kind.


Hopefully creating new connections won't be prohibitively expensive for your needs and purposes so that you can make your pool that way.

huangapple
  • 本文由 发表于 2023年5月14日 09:16:29
  • 转载请务必保留本文链接:https://go.coder-hub.com/76245442.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定