[Cpan-forum-commit] rev 148 - in branches/db_changes: . bin
lib/CPAN lib/CPAN/Forum
svn at pti.co.il
svn at pti.co.il
Fri Apr 22 17:26:22 IDT 2005
Author: gabor
Date: 2005-04-22 17:26:21 +0300 (Fri, 22 Apr 2005)
New Revision: 148
Added:
branches/db_changes/bin/recent.pl
Modified:
branches/db_changes/TODO
branches/db_changes/bin/populate.pl
branches/db_changes/lib/CPAN/Forum.pm
branches/db_changes/lib/CPAN/Forum/Groups.pm
Log:
some older changes
Modified: branches/db_changes/TODO
===================================================================
--- branches/db_changes/TODO 2005-03-26 14:36:57 UTC (rev 147)
+++ branches/db_changes/TODO 2005-04-22 14:26:21 UTC (rev 148)
@@ -3,7 +3,19 @@
(both RSS and e-mail)
+BUG:
+When I try to reply and the original subject is already 50 chars long
+in offers a new subject with Re: prefix but then when I try to submit
+it won't let me.
+(So far is ok, though it should 70 or 100 long)
+
+But the main problem is that if I delete the last word from the
+subject and press preview again
+it returns the same error message as it put back the word where it was earlier.
+
+
+
- Clean up the listing of the subscribers at the bottom of the pages:
Separate people who will receive all messages
@@ -55,7 +67,11 @@
+In order to avoid accepting postings today that will break when we add more
+tags, we will reject any submission that is not correctly marked up.
+
+
- Create larger discussion groups (e.g. Web development and All)
@@ -148,3 +164,115 @@
and its server.
+- Decide on Basic Markup language and how to extend for shortcuts opening tag
+for code: <code[^>]*> but right now only <code> should be accepted closing
+tag for code: </code>
+
+- check all submitted fields (restrict posting size to 10.000 Kbyte ?
+- Improve text and explanations.
+
+- clean up documentation
+
+- add indexes to the tables ?
+
+- show the release dates and version numbers of the modules
+
+Authentication and user management process:
+- new user comes to our site we give him a cookie, when he wants to login we offer him
+-- login using the auth.perl.org credentials
+-- login using XYZ credentials
+-- create local credential
+
+-- For auth.perl.org
+--- redirect the user to auth.perl.org wait till he logs in there (maybe even creates the new account)
+--- sets the preferences
+--- comes back
+--- we can fetch some of the information from that user
+--- we need to keep the user_id received from auth.perl.org for later identification of the user
+--- while we tell the user we would like to get the username/fullname/e-mail
+address from auth he might not want to give, for this case we should have our
+way to update the locally updated username, full name and validated e-mail
+address.
+
+-- For local credentials we need the user to give us
+username/password/fullname and validated e-mail address.
+
+
+We have to make sure that usernames which are displayed don't collide. Maybe we
+should use separate fields for usernames from various sources and when
+displayed we might prefix it auth:gabor, local:gabor etc. Not nice, any better
+way ?
+
+- Fix Installation
+
+- when installing one might need to be root, in order to set the permissions
+correctly ?
+
+- as user www fetch the module list file, unzip it in the db directory (as
+this is the only directory we can write to) and run the populator
+
+- on a new installation, change the ownership of directories (or at least tell
+the user to do so)
+
+- Write comprehensive test suit
+
+- Reply within a thread
+
+When replying to a post within a thread we might want to open the editor window
+in the middle of the thread, just below the post I am responding to.
+
+- make sure links that are relevant for distros don't show up on pages which
+don't belong to distros. (e.g. a link to search.cpan.org/dist/CGI is ok but a
+link to search.cpan.org/dist/General is not)
+
+- Sometime we'll want to post a message in more than one group, e.g. now I'd
+like to know how to use CGI::Session with DBD::SQLite. I might want to post the
+message on more than one list at the same time as this is related to more than
+one module. Porbably if I need to chose one I'll select CGI::Session as I am
+trying to use that but it might be a nice feature. Maybe I need to tell one
+module as the main group and then have a way to associate a few more modules
+with the posting.
+
+This can be done by de-coupleing the name of the distribution from the posts table for all the distributions or we can add such an extra table for the additional distributions so there will be a leading distro of the thread.
+
+
+- Create a group for
+- each Distribution (DONE)
+- Some bigger groups (eg. databases, testing, )
+maybe put each distribution under one or more of the groups too
+- General and other special purpose groups such as News (for the site)
+where only "administrators" can post.
+
+I am not sure if I have to keep all these things in one table and if the
+same form has to serve for creating messages in both distros and categories.
+
+- Database or plain files ?
+
+I think every information should be in the database but then we might want to
+generate static pages from the posts and discussions in order to reduce the
+need to fetch information from the database. Hmm, it sound faster but we'll
+probabl want to build the pages on the fly anyway so maybe it does not improve
+anything. We can start off by totally dynamic pages and then see if making them
+static will reduce the load on the server. First we'll have to have load on the
+server. :-)
+
+- Check if the technique we use to remember the last request before login
+cannot cause some security problem such as remembering the last request of
+someone else who used the same machine recently ?
+
+- xml - provided
+
+
+
+Shlomi:
+The Forum uses cgiapp_prerun to set the mode according to the PATH_INFO instead of
+using a mode_param code-reference. This causes a lot of warnings in the logs,
+and doesn't really belong in cgiapp_prerun.
+
+It cannot be hosted on a URL except for its own virtual host, as it uses
+absolute URLs. ("/login/", "/register/", etc.) A better idea would be to
+track the path that the web-server gives (it's in one of the environment
+variables) and then to construct a /cpan-forum/login/ /cpan-forum/register/
+etc. path. (or use relative URLs).
+
+
Modified: branches/db_changes/bin/populate.pl
===================================================================
--- branches/db_changes/bin/populate.pl 2005-03-26 14:36:57 UTC (rev 147)
+++ branches/db_changes/bin/populate.pl 2005-04-22 14:26:21 UTC (rev 148)
@@ -59,43 +59,34 @@
# skip scripts
next if not $d->prefix or $d->prefix =~ m{^\w/\w\w/\w+/scripts/};
-
my $name = $d->dist;
- my %new = (
- version => ($d->version() || ""),
- pauseid => ($d->cpanid() || ""),
- );
-
if (not $name) {
#warn "No name: " . $d->prefix . "\n";
next;
}
# for now skip names that start with lower case
- next if $name =~ /^[a-z]/;
-
+ #next if $name =~ /^[a-z]/;
+
+ my %new = (
+ version => ($d->version() || ""),
+ pauseid => ($d->cpanid() || ""),
+ );
+
+
my ($g) = CPAN::Forum::Groups->search(name => $name);
if ($g) {
- #if (not defined $version{$name} or $version{$name} ne $new_version) {
- # $version{$name} ||= ""; # to avoid warnings;
- # # send e-mail to whoever asked for it.
- # $version_message .= "The version of $name has changed from $version{$name} to $version\n";
- #}
- #if (not defined $version{$name} or $version{$name} ne $new_version) {
- # #$pausid_message .= "The PAUSEID of $name has changed from $old_paused to $new_pauseid\n";
- #}
-
my $changed;
foreach my $field (qw(version pauseid)) {
- print "$name\n";
- print "NEW: $new{$field}\n";
- print "OLD: " . $g->$field, "\n";
- <STDIN>;
+ #print "$name\n";
+ #print "NEW: $new{$field}\n";
+ #print "OLD: " . $g->$field, "\n";
+ #<STDIN>;
if (not defined $g->$field or $g->$field ne $new{$field}) {
- print "change\n";
+ #print "change\n";
+ $message{$field} .= sprintf "The %s of %s has changed from %s to %s\n",
+ $field, $name, ($g->$field || ""), $new{$field};
$g->$field($new{$field});
- $message{$field} .= sprintf "The $field of %s has changed from to %s\n",
- $name, ($g->$field || ""), $new{$field};
$changed++;
}
}
Added: branches/db_changes/bin/recent.pl
===================================================================
--- branches/db_changes/bin/recent.pl 2005-03-26 14:36:57 UTC (rev 147)
+++ branches/db_changes/bin/recent.pl 2005-04-22 14:26:21 UTC (rev 148)
@@ -0,0 +1,127 @@
+#!/usr/bin/perl
+use strict;
+use warnings;
+
+
+# client for processing the most recent
+# PAUSE uploads ftp://www.cpan.org/modules/01modules.mtime.rss
+#
+
+
+use FindBin qw ($Bin);
+use LWP::Simple qw(getstore);
+use XML::RSS;
+use CPAN::DistnameInfo;
+use Getopt::Long qw(GetOptions);
+use lib "lib";
+use CPAN::Forum::INC;
+
+my $dir = "$Bin/../db";
+my $dbfile = "$dir/forum.db";
+
+my %opts;
+
+CPAN::Forum::DBI->myinit($dbfile);
+
+GetOptions(\%opts, "sendmail", "file=s");
+
+
+my $remote_file = "http://www.cpan.org/modules/01modules.mtime.rss";
+my $local_file = $opts{file};
+
+if (not $local_file or not -e $local_file) {
+ $local_file = "db/01modules.mtime.rss";
+ print "Fetching $remote_file\n";
+ getstore $remote_file, $local_file;
+}
+
+
+my $rss = XML::RSS->new();
+$rss->parsefile($local_file);
+
+my %message = (
+ version => "",
+ pauseid => "",
+ news => "",
+);
+
+foreach my $item (reverse @{$rss->{items}}) {
+ my $link = $item->{link};
+ $link =~ s{^http://www.cpan.org/modules/by-authors/}{authors/};
+ my $d = CPAN::DistnameInfo->new($link);
+ #print $link, "\n";
+ #print $d->dist, "\n";
+ #print $d->version, "\n";
+ #print $d->cpanid(), "\n";
+
+ my $name = $d->dist();
+ my %new = (
+ version => ($d->version() || ""),
+ pauseid => ($d->cpanid() || ""),
+ );
+
+ my ($g) = CPAN::Forum::Groups->search(name => $name);
+ if ($g) {
+ my $changed;
+ foreach my $field (qw(version pauseid)) {
+ #print "$name\n";
+ #print "NEW: $new{$field}\n";
+ #print "OLD: " . $g->$field, "\n";
+ #<STDIN>;
+ $new{version} =~ s/\.?0*$//; # so it won't try to update numbers with 00 or . endings.
+ if (not defined $g->$field or $g->$field ne $new{$field}) {
+ #print "change\n";
+ $message{$field} .= sprintf "The %s of %s has changed from %s to %s\n",
+ $field, $name, ($g->$field || ""), $new{$field};
+ $g->$field($new{$field});
+ $changed++;
+ }
+ }
+
+ $g->update if $changed;
+ next;
+ }
+
+ $message{news} .= sprintf "%s %s\n", $name, $new{version}, $new{pauseid};
+ eval {
+ my $g = CPAN::Forum::Groups->create({
+ name => $name,
+ gtype => $CPAN::Forum::DBI::group_types{Distribution},
+ version => $new{version},
+ pauseid => $new{pauseid},
+ });
+ };
+ if ($@) {
+ warn "$name\n";
+ warn $@;
+ }
+}
+
+my %mail = (
+ To => 'gabor at pti.co.il',
+ From => 'cpanforum at cpanforum.com',
+ Subject => 'CPAN Version Update',
+ Message => $message{version},
+);
+if ($opts{sendmail}) {
+ sendmail(%mail);
+} else {
+ open my $fh, ">", "$Bin/../cpan_version_update";
+ print $fh $message{version};
+}
+
+%mail = (
+ To => 'gabor at pti.co.il',
+ From => 'cpanforum at cpanforum.com',
+ Subject => 'New CPAN Distros',
+ Message => $message{news},
+);
+if ($opts{sendmail}) {
+ sendmail(%mail);
+} else {
+ open my $fh, ">", "$Bin/../cpan_new_distros";
+ print $fh $message{news};
+}
+
+
+
Modified: branches/db_changes/lib/CPAN/Forum/Groups.pm
===================================================================
--- branches/db_changes/lib/CPAN/Forum/Groups.pm 2005-03-26 14:36:57 UTC (rev 147)
+++ branches/db_changes/lib/CPAN/Forum/Groups.pm 2005-04-22 14:26:21 UTC (rev 148)
@@ -14,5 +14,7 @@
__PACKAGE__->set_sql(count_like => "SELECT count(*) FROM __TABLE__ WHERE %s LIKE '%s'");
__PACKAGE__->set_sql(count => "SELECT count(*) FROM __TABLE__ WHERE %s = '%s'");
+#use Data::Dumper;
+#__PACKAGE__->add_trigger(before_update => sub {warn Dumper $_[0]});
1;
Modified: branches/db_changes/lib/CPAN/Forum.pm
===================================================================
--- branches/db_changes/lib/CPAN/Forum.pm 2005-03-26 14:36:57 UTC (rev 147)
+++ branches/db_changes/lib/CPAN/Forum.pm 2005-04-22 14:26:21 UTC (rev 148)
@@ -243,56 +243,14 @@
=head2 TODO
-- Decide on Basic Markup language and how to extend for shortcuts opening tag
-for code: <code[^>]*> but right now only <code> should be accepted closing
-tag for code: </code>
-
-- check all submitted fields (restrict posting size to 10.000 Kbyte ?
-- Improve text and explanations.
-
-clean up documentation
-
-add indexes to the tables ?
-
-show the release dates and version numbers of the modules
-
-Authentication and user management process:
-- new user comes to our site we give him a cookie, when he wants to login we offer him
--- login using the auth.perl.org credentials
--- login using XYZ credentials
--- create local credential
-
--- For auth.perl.org
---- redirect the user to auth.perl.org wait till he logs in there (maybe even creates the new account)
---- sets the preferences
---- comes back
---- we can fetch some of the information from that user
---- we need to keep the user_id received from auth.perl.org for later identification of the user
---- while we tell the user we would like to get the username/fullname/e-mail
-address from auth he might not want to give, for this case we should have our
-way to update the locally updated username, full name and validated e-mail
-address.
-
--- For local credentials we need the user to give us
-username/password/fullname and validated e-mail address.
-
-
-We have to make sure that usernames which are displayed don't collide. Maybe we
-should use separate fields for usernames from various sources and when
-displayed we might prefix it auth:gabor, local:gabor etc. Not nice, any better
-way ?
-
Subject field:
-- <= 50 chars
+- <= 80 chars
- Can contain any characters, we'll escape them when showing on the web site
Text field:
- No restriction on line length, let the HTML handle that part
- The text is divided into areas of free text and marked sections
-In order to avoid accepting postings today that will break when we add more
-tags, we will reject any submission that is not correctly marked up.
-
- Pages:
new mesage: EDITOR; PREVIEW + EDITOR
show: POST
@@ -380,92 +338,6 @@
In addition when displaying the list of all the messages to a specific module, logged in users
will see their current subscription to this module (even if that is empty).
-
-- Fix Installation
-
-- when installing one might need to be root, in order to set the permissions
-correctly ?
-
-- as user www fetch the module list file, unzip it in the db directory (as
-this is the only directory we can write to) and run the populator
-
-- on a new installation, change the ownership of directories (or at leas tell
-the user to do so)
-
-
-- Write comprehensive test suit
-
-- Reply within a thread
-
-When replying to a post within a thread we might want to open the editor window
-in the middle of the thread, just below the post I am responding to.
-
-- make sure links that are relevant for distros don't show up on pages which
-don't belong to distros. (e.g. a link to search.cpan.org/dist/CGI is ok but a
-link to search.cpan.org/dist/General is not)
-
-- Sometime we'll want to post a message in more than one group, e.g. now I'd
-like to know how to use CGI::Session with DBD::SQLite. I might want to post the
-message on more than one list at the same time as this is related to more than
-one module. Porbably if I need to chose one I'll select CGI::Session as I am
-trying to use that but it might be a nice feature. Maybe I need to tell one
-module as the main group and then have a way to associate a few more modules
-with the posting.
-
-This can be done by de-coupleing the name of the distribution from the posts table for all the distributions or we can add such an extra table for the additional distributions so there will be a leading distro of the thread.
-
-
-- Getting the listing of all ~8000 module names takes a long time.
-I should profile it.
-1) write a small script that will run the relevant code on the command line,
-2) time this
-3) look at the size of the output 386K -> it won't fly, you can't have such a page
-on the web. Other solutions:
-- type in the name
-- search for the name
-
-
-- Create a group for
-- each Distribution (DONE)
-- Some bigger groups (eg. databases, testing, )
-maybe put each distribution under one or more of the groups too
-- General and other special purpose groups such as News (for the site)
-where only "administrators" can post.
-
-I am not sure if I have to keep all these things in one table and if the
-same form has to serve for creating messages in both distros and categories.
-
-- Database or plain files ?
-
-I think every information should be in the database but then we might want to
-generate static pages from the posts and discussions in order to reduce the
-need to fetch information from the database. Hmm, it sound faster but we'll
-probabl want to build the pages on the fly anyway so maybe it does not improve
-anything. We can start off by totally dynamic pages and then see if making them
-static will reduce the load on the server. First we'll have to have load on the
-server. :-)
-
-- Check if the technique we use to remember the last request before login
-cannot cause some security problem such as remembering the last request of
-someone else who used the same machine recently ?
-
-- xml - provided
-
-- favicon.ico and a banner image would be good
-
-
-Shlomi:
-The Forum uses cgiapp_prerun to set the mode according to the PATH_INFO instead of
-using a mode_param code-reference. This causes a lot of warnings in the logs,
-and doesn't really belong in cgiapp_prerun.
-
-It cannot be hosted on a URL except for its own virtual host, as it uses
-absolute URLs. ("/login/", "/register/", etc.) A better idea would be to
-track the path that the web-server gives (it's in one of the environment
-variables) and then to construct a /cpan-forum/login/ /cpan-forum/register/
-etc. path. (or use relative URLs).
-
-
=head2 TEMPLATES
@@ -517,12 +389,6 @@
=head1 METHODS
-=head2 cgiapp_init
-
-Standard CGI::Application method.
-
-Setup the Session object and the default HTTP headers
-
=cut
sub cgiapp_init {
@@ -553,7 +419,6 @@
$self->log->debug("Cookie received: " . $self->query->cookie($cookiename) );
CGI::Session->name($cookiename);
$self->session_config(
- #CGI_SESSION_OPTIONS => [ "driver:File", $self->query, {Directory => "/tmp"}],
CGI_SESSION_OPTIONS => [ "driver:SQLite", $self->query, {Handle => $dbh}],
COOKIE_PARAMS => {
-expires => '+24h',
@@ -569,7 +434,6 @@
# on the other hand it is needed in Opera to make sure it won't cache pages.
-charset => "utf-8",
);
- #$self->session_cookie();
}
sub _set_log_level {
@@ -595,28 +459,29 @@
}
# modes that can be accessed without a valid session
-my @free_modes = qw(home
- pwreminder pwreminder_process
- login login_process
- register register_process
- logout
- about faq
- posts threads dist users
- search all
- site_is_closed
- help
- rss );
+my @free_modes = qw(
+ home
+ pwreminder pwreminder_process
+ login login_process
+ register register_process
+ logout
+ about faq
+ posts threads dist users
+ search all
+ site_is_closed
+ help
+ rss );
my @restricted_modes = qw(
- new_post process_post
- mypan
- admin
- admin_process
- admin_edit_user
- admin_edit_user_process
- add_new_group
- response_form
- module_search
- selfconfig change_password change_info update_subscription);
+ new_post process_post
+ mypan
+ admin
+ admin_process
+ admin_edit_user
+ admin_edit_user_process
+ add_new_group
+ response_form
+ module_search
+ selfconfig change_password change_info update_subscription);
my @urls = qw(
logout
@@ -632,11 +497,6 @@
mypan selfconfig
search all rss);
-=head2 setup
-
-Regular CGI::Appication method to setup the list of all run modes and the default run mode
-
-=cut
sub setup {
my $self = shift;
$self->start_mode("home");
@@ -646,8 +506,6 @@
=head2 cgiapp_prerun
-Regular CGI::Application method
-
We use it to change the run mode according to the requested URL (PATH_INFO).
Maybe we should move his code to the mode_param method ?
@@ -670,9 +528,9 @@
if (not $rm or $rm eq "home") {
if ($ENV{PATH_INFO} =~ m{^/
- ([^/]+) # first word till after the first /
- (?:/(.*))? # the rest, after the (optional) second /
- }x) {
+ ([^/]+) # first word till after the first /
+ (?:/(.*))? # the rest, after the (optional) second /
+ }x) {
my $newrm = $1;
my $params = $2 || "";
if (grep {$newrm eq $_} @urls) {
@@ -687,7 +545,6 @@
# shall I make more noise ?
}
}
-
}
$self->log->debug("Current runmode: $rm");
More information about the Cpan-forum-commit
mailing list