Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
MrMoo
Sep 14, 2000

Is it possible to SEO-style normalise a URI in a MySQL stored function? I'm thinking of a function that converts "Lingeries sets" into "lingerie-sets", and an appropriate method of querying the database for reverse lookup.

Normalising is easy on the command line:
code:
echo -n "Lingerie Sets" | tr "A-Z" "a-z" | tr -cs "a-z" "-"
lingerie-sets
And the SQL for the reverse lookup seems simple:
code:
SELECT *
FROM   `CATEGORIES`
WHERE  `NAME` REGEXP REPLACE('lingerie-sets','-','.+')
(edit) Maybe better not to strip characters to support other languages, hence something like this:

code:
DROP FUNCTION `sf_uri_normalise`//
CREATE DEFINER=`root`@`localhost` FUNCTION `sf_uri_normalise`(s VARCHAR(100))
    RETURNS varchar(100) CHARSET utf8
    NO SQL
    DETERMINISTIC
    COMMENT 'normalise a string to a SEO-friendly uri'
BEGIN
        RETURN LOWER(REPLACE(s, ' ', '-'));
END

MrMoo fucked around with this message at 20:05 on Jan 30, 2009

Adbot
ADBOT LOVES YOU

MrMoo
Sep 14, 2000

moostaffa posted:

I currently have all this data in one table (No expensive queries will be needed on this, only lookups on the single index), would it give any performance boost to split this table into, say, 10 or 100 smaller tables?

On high end databases no, it will make it worse. The database' job is to manage indexes and large tables. However I think only Oracle is good at this. Sybase used to be terrible unless things have changed.

MrMoo
Sep 14, 2000

fletcher posted:

This was with mysqld running on my localhost and /tmp/test_data.csv is also on my local machine, shouldn't the FILE permission cover that?

It is covered in the docs:

quote:

Using LOCAL is a bit slower than letting the server access the files directly, because the contents of the file must be sent over the connection by the client to the server. On the other hand, you do not need the FILE privilege to load local files.
http://dev.mysql.com/doc/refman/5.0/en/load-data.html

And it has its own page on security issues.

MrMoo
Sep 14, 2000

How does MariaDB manage variables? I'm seeing this unexpected case that a key is not being used when a specifying a local variable.
SQL code:
MariaDB [trth]> explain SELECT max(bid_price) AS bid_price, exchange FROM quotes WHERE quotes.ric = 'A' AND timestamp 
BETWEEN @start_time AND @end_time GROUP BY exchange ORDER BY NULL;
+------+-------------+--------+-------+---------------+---------------+---------+------+------+------------------------------+
| id   | select_type | table  | type  | possible_keys | key           | key_len | ref  | rows | Extra                        |
+------+-------------+--------+-------+---------------+---------------+---------+------+------+------------------------------+
|    1 | SIMPLE      | quotes | range | ric_timestamp | ric_timestamp | 9       | NULL |  659 | Using where; Using temporary |
+------+-------------+--------+-------+---------------+---------------+---------+------+------+------------------------------+
1 row in set (0.00 sec)
  
MariaDB [trth]> set @trade_ric ='A';
Query OK, 0 rows affected (0.00 sec)

MariaDB [trth]> explain SELECT max(bid_price) AS bid_price, exchange FROM quotes WHERE quotes.ric = @trade_ric AND timestamp
 BETWEEN @start_time AND @end_time GROUP BY exchange ORDER BY NULL;
+------+-------------+--------+------+---------------+------+---------+------+----------+------------------------------+
| id   | select_type | table  | type | possible_keys | key  | key_len | ref  | rows     | Extra                        |
+------+-------------+--------+------+---------------+------+---------+------+----------+------------------------------+
|    1 | SIMPLE      | quotes | ALL  | NULL          | NULL | NULL    | NULL | 10841597 | Using where; Using temporary |
+------+-------------+--------+------+---------------+------+---------+------+----------+------------------------------+
1 row in set (0.00 sec)
There is a half-explanation for T-SQL: http://www.sqlbadpractices.com/using-local-variables-in-t-sql-queries/

MrMoo fucked around with this message at 20:03 on Dec 4, 2014

MrMoo
Sep 14, 2000

Ugh, apparently character set hell. An outstanding bug, workaround is to force the same character set everywhere.

http://stackoverflow.com/q/16752922/175849

The TokuDB storage engine is nice though, 100GB source data down to 10GB table data and only 2GB compressed.

MrMoo fucked around with this message at 23:18 on Dec 5, 2014

MrMoo
Sep 14, 2000

There is clearly something special with the backup process, find out what and how it as affecting the SELECT query. It is not normal, but you may have a bug in your version of MySQL too :toot:

MrMoo
Sep 14, 2000

Well for starters you can clean up that query using the IN expression, secondly it looks like you simply want a ranking function but this is not standard SQL and thus depends upon your database.

MrMoo
Sep 14, 2000

Can you update this SQL Fiddle with your intent?

MrMoo
Sep 14, 2000

Is there a standard solution for supporting a dynamic query of form a OR b AND (c OR d) from some proprietary interface to SQL? It seems that some limits are required to stop forcing full table scans. Table along the lines of:

code:
article_id |  code
-----------+-------
  100      |  one
  100      |  two
  200      |  one
  300      |  three
Find articles with dynamic queries, e.g. code = one AND (code = two OR code = three)

edit: It looks like using the MySQL family I can use the MATCH() ... AGAINST() function suite if I prepare all the codes into a FULLTEXT index per row (article_id above). Then that provides the full boolean mode used in text searches.

edit 2: Is this only possible with a "search engine"? I'm looking at Sphinx and the problem appears to come down to performing boolean matching on what they term multi-valued attributes (MVAs). Sphinx appears cute as can integrate with MariaDB to be driven by SQL.

http://stackoverflow.com/questions/13643114/how-do-i-search-for-mvas-with-sphinxse
https://mariadb.com/kb/en/mariadb/sphinx-storage-engine/

MrMoo fucked around with this message at 13:58 on Jul 22, 2015

MrMoo
Sep 14, 2000

I think Percona is using the functionality from TokuDB which supports a lot of online changes with background rebuilds. I certainly recommend looking at MariaDB + TokuDB.

MrMoo
Sep 14, 2000

Couldn't you summon a super view over the set of tables?

I would be surprised if Oracle does not have something nice for this configuration, the MySQL family have been slowly adding multi-partition and multi-table engines so a single query is replicated across each.

MrMoo
Sep 14, 2000

I want to say w3m and a website but I would hope there is something native that works with ncurses and is less retarded.

MrMoo
Sep 14, 2000

For data loading have a gander at MemSQL, it's something crazy like 1 million rows per second loading data. MariaDB + TokuDB has some good optimizations on empty tables too.

MrMoo
Sep 14, 2000

Stupid question ahoy. If I have a simple soft delete query, i.e.
SQL code:
UPDATE assets SET is_deleted WHERE id = ?;
I can perform the basic "zero-write optimisation" as such,
SQL code:
UPDATE assets SET is_deleted WHERE id = ? AND is_deleted = 0;
However if I want the caller to be able to distinguish between whether the row exists and whether it was already soft deleted, is there a recommended form for that? In pseudo SQL that could be:
SQL code:
UPDATE assets SET is_deleted WHERE id = ? AND is_deleted = 0;
CASE WHEN (changes() = 0) THEN
  SELECT COUNT(*) AS is_exists, is_deleted FROM assets WHERE id = ?
ELSE
  SELECT 1 AS is_exists, 1 AS is_deleted
END;
I'm basically writing some unit tests and seeing whether I can raise HTTP/404 and HTTP/410 as separate responses. The non-transaction safe form would be to query on the asset ID if the UPDATE statement returned zero changes.

Currently targeting SQLite, open to learn for other vendor dialects.

The obvious answer is to not bother making the distinction, but it's nice to see if it is possible and practical.

MrMoo fucked around with this message at 04:11 on Oct 7, 2023

MrMoo
Sep 14, 2000

The problem appears to be more about reading a column value before an UPDATE is applied. With Postgres this is possible, but Sqlite likes to only return the updated value.

https://stackoverflow.com/questions/7923237/return-pre-update-column-values-using-sql-only

i.e. the following is fine in PGSQL:
SQL code:
UPDATE assets SET is_deleted = 1 WHERE id = 1
RETURNING (SELECT is_deleted FROM assets WHERE id = 1);

MrMoo
Sep 14, 2000

Alas, sqlite doesn't like that for some reason, need to check why, https://www.sqlite.org/lang_update.html

SQL code:
UPDATE demo AS x
SET name = "frog"
FROM demo AS y
WHERE x.id = y.id
    AND x.id = 3
RETURNING y.name AS old_name;
https://sqliteonline.com/

Raises "no such column: y.name" 🤷‍♀️ Doesn't seem to understand table alias in the RETURNING clause.

There are three conditions:
  • If id matches and is not already soft-deleted. This is detectable via changes() = 1 if there is the additional WHERE is_deleted = 0 clause.
  • If id matches and is already soft-deleted. The basic logic would be that changes() = 0 then verify that is_deleted = 1. Otherwise the above PGSQL syntax of detecting the value of is_deleted before the UPDATE would achieve the same goal.
  • If id does not match any rows, i.e. changes() = 0 and a SELECT COUNT(*) FROM assets WHERE id = 1 would be zero.

I think that just leaves this:
SQL code:
BEGIN TRANSACTION;
SELECT name FROM demo WHERE id = 3;
UPDATE demo SET name = "frog" WHERE id = 3 AND name <> "frog";
COMMIT;
Remembering that this version still returns "frog":
SQL code:
UPDATE demo
SET name = 'frog'
WHERE id = 3 AND name <> 'frog'
RETURNING name;
Which I guess is explained by this:

quote:

The first prototype of the RETURNING clause returned values as they were generated. That approach used less memory, but it had other problems:

If the calls sqlite3_step() for two or more DML statements where interleaved and if one of the statements hit a constraint failure and aborted, reverting its changes, then that could disrupt the operation of the other DML statement. This could not corrupt the database file, but it could yield surprising and difficult-to-explain results in the database.

If an application failed to call sqlite3_step() repeatedly until it received SQLITE_DONE, then some of the database changes might never occur.

The order of operations was different from client/server database engines like PostgreSQL, which might cause portability issues for some applications.

For these reasons, the current implementation was modified so that all database changes happen before any RETURNING output is emitted.
https://www.sqlite.org/lang_returning.html

MrMoo fucked around with this message at 18:28 on Oct 7, 2023

MrMoo
Sep 14, 2000

It's in England though?

https://sqlbits.com/news/sqlbits-2024-unveiled/

Adbot
ADBOT LOVES YOU

MrMoo
Sep 14, 2000

GigaPeon posted:

SQL Prompt

Flyway

Flyway is pretty nice, although you have to do the work to make it powerful.



That is an impressive choice of styling.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply