Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
kiwid
Sep 30, 2013

Does anyone have any recommended reading regarding database design best practices?

I'm beginning the foundation of a new project I'm working on and I have complete freedom in how I design the database for this application. I've designed databases before but generally just built things with lots of surrogate keys, no foreign constraints, etc. I want to build this database with best practices in mind. For example, I don't know whether to use natural/business keys vs surrogate keys, simple keys vs compound/composite keys (identifying vs non-identifying relationships), etc.

kiwid fucked around with this message at 20:30 on Oct 30, 2014

Adbot
ADBOT LOVES YOU

kiwid
Sep 30, 2013

I have an application where there are several admins managing the administrative tables.

I want to start tracking what admins made what changes at what time.

What is the best way to do this? I take it I could store everything in one table? user_id, current_timestamp, etc. But how do I specify what change is taking place? Anyone have any examples?

Another way I was thinking would be to use soft deletes on all the tables I want to track, but that would mean putting these columns on every table (deleted_at, deleted_by, updated_at, updated_by) and I wouldn't really be able to tell what the changes were.

kiwid fucked around with this message at 19:00 on Nov 5, 2014

kiwid
Sep 30, 2013

Awesome, I'll check out triggers. Thanks.

kiwid
Sep 30, 2013

If you have to store a test result, and this test result could be either a few word sentence (varchar), or a count (integer), or a percentage (decimal?), but never be all three, only 1 of the three. How would you store it?

kiwid
Sep 30, 2013


MySQL

edit: Here is how I'm currently doing it. Is this a good way to do it?

code:
test_dec decimal(4,1)  NULL DEFAULT NULL
test_var varchar(48)   NULL DEFAULT NULL
test_int integer(11)   NULL DEFAULT NULL
Selecting from the database:
code:
SELECT CONCAT_WS("", `table`.`test_dec`, `table`.`test_var`, `table`.`test_int`) AS `test`
FROM `table`
Inserting into the database (pseudocode):
code:
if is_int($test)
    $column = 'test_int';
elseif is_numeric($test)
    $column = 'test_dec';
else
    $column = 'test_var';

insert into `table` values($column) $test

kiwid fucked around with this message at 03:28 on Nov 7, 2014

kiwid
Sep 30, 2013

More questions!

Looking at these two schema's, which one is the proper way to build the relationships?





Each category can have many commodities and each commodity can be in many categories.

Each facility can have multiple categories and commodities but does not have access to all categories or all commodities.

Each category can have many facilities.

Each commodity can have many facilities.

An example of a category is "Food" or "Seed". An example of a commodity is "Corn", "Soybeans", or "Wheat".

Soybeans can both be a food or a seed for example, hence the category_has_commodity table.

Some facilities can only process seed, not food, but will still have access to the soybeans commodity. Some facilities can process food and seed, and multiple commodities.

kiwid fucked around with this message at 22:25 on Nov 7, 2014

kiwid
Sep 30, 2013

Some facilities have access to some categories (maybe all but never none), and some commodities (maybe all but never none).

If a facility has access to a category, they do not necessarily have access to all commodities in that category, and vice versa, if a facility has access to a commodity, they may not have access to all categories that that commodity belongs to.

Does that mean I should be using the second schema?

kiwid
Sep 30, 2013

Like this?



I didn't even realize you could do that, I thought you had to join two tables to one many to many. Didn't realize you can do a many-to-many for three tables.

kiwid
Sep 30, 2013

PhantomOfTheCopier posted:

I don't know that I can answer without sounding utterly biased. First, I knew you were going to say MySQL... probably based on the wording in the original question. In all honesty, you might as well just make a single varchar(random number) like every other MySQL user and call it good, because everything is likely to be handled, ultimately, in the application, so why bother with all that noise in the database if it doesn't gain you anything.

Now then, one choice is to store the data that way, or encoded in such a way that the application does most of the interpretation with no effect on database performance because you aren't running optimized queries over those fields anyway. Encoding could be anything from a blob to custom formats, markup, etcetera. This pushes all the integrity off to the application. Indeed, one of the main issues with your proposal is integrity: Nothing prevents all three from being non-null.

As there is no programmatic union (a la C, "store multiple things in the same space"), you either need to use a database that's custom and gives it to you, or, lacking that, if you decide you must have a union type, you can create a custom data type. In that case, as in your proposed structure, you'd really need to store another field identifying the actual type. This is no different than a union, where the data stored is fundamentally useless for decoding purposes unless you know whether it's an integer, float, character, pointer, whatever. Presumably, you would then be able to select on specific types, or wrap CASE statements around each field to enforce NULL based on the registered type for that row. Even if the application "accidentally" loaded (different) values into all three fields, you'd have a unique result for the row.

A third approach is to separate the fields into their own tables, only inserting a referenced row into the one appropriate table. While this provides some optimization on queries for subtypes, it seems a bit ridiculous for the application you've described. It seems highly unlikely that your primary use case is "searching for test_dec between 0.73 and 0.81", whence it's unlikely that you'd even much care about indexing. If all queries you perform are already restricted to groups/classes/quarters, a table scan of the relevant rows will be fast enough.

Please don't tell us this was a homework question.

Definitely isn't a home work question :)

This is more of a personal learning project that happened to get put into Protoduction. The application is actually quite large and it's going to be expanded really soon. What' I've shown in this thread is really only a small fraction of the app and database. What I'm doing now is basically refactoring the application since I've learnt better programming practices and I'm trying to learn better database design so that in a couple months, when the planned expansion to this application starts being defined, I have a better base to work off of.

I think your third approach sounds interesting and I would be searching for values that are out of the min and max specifications so thanks for giving me these options to look into.

Curious though, why do you have a bias against MySQL? I assume you're a MSSQL guy? I really don't know much about other database platforms, is there a reason MySQL shouldn't be used?

kiwid fucked around with this message at 17:35 on Nov 9, 2014

kiwid
Sep 30, 2013

Hmm, I never realized that. Why is it the default in a LAMP/LEMP stack and why is it so popular?

kiwid
Sep 30, 2013

We do have MSSQL in the office but it's 2005. This place tends to severely lag behind in purchasing software and licensing. So it looks like I'll start looking into PostgreSQL.

kiwid
Sep 30, 2013

I'm creating a internal webapp that tracks a couple different types of service requests that users fill out.

The difference between the 3 different types of service requests is that they all share a lot of duplicate fields and then have a some fields that are specific to that type.

For example:

request type 1 fields
customer name
customer code
customer address
... a bunch of other customer fields
sample type
sampled date
sample count

request type 2 fields
customer name
customer code
customer address
... a bunch of other customer fields
flight date
pilot
flight reason

request type 3 fields
customer name
customer code
customer address
... a bunch of other customer fields
allocated personal
items purchased
total

Currently, I've created separate tables for each service request type but now I'm thinking I should just track everything in one table and have a one-to-one relationship with 3 other tables that each contain the additional fields that are not duplicated.

Is this a better way to do this?

kiwid
Sep 30, 2013

Jethro posted:

One service request table with three supplemental tables is better than three service request tables, but if it's just a handful of extra fields, then you could also just have one table with all the fields and then just leave them empty for the service requests that don't need them.

Thanks.

Yeah the example I used I left out a lot of details but there can be between 10-15 additional fields so I'll just go with the 1 main table and 3 additional tables.

kiwid
Sep 30, 2013

Does MySQL allow a unique index between several keys that still allows NULL?

For example:

Columns:
user_id NOT NULL
account_id NOT NULL
type_id NULL

unique('user_id', 'account_id', 'type_id')

This would enforce that there can only be one user with an account where type = whatever

However, since type can be NULL, there still needs to allow for a user to own multiple accounts where type is NULL.

Is this possible?

Edit: Oh nevermind, apparently this is baked in already. I was getting confused because I tried looking it up before implementing it and found this bug reported in 2005 and had no idea what happened.

kiwid fucked around with this message at 20:58 on Oct 28, 2015

kiwid
Sep 30, 2013

Database design question.

I have two entities, a users table, and an accounts table.

A user can "own" many accounts and an account can belong to many "owners".
A user can "manage" many accounts but an account can only have one "manager" which is a user.
A user can be the "sales lead" for many accounts but an account can only have one "sales lead" which is a user.
A user can be the "consultant" for many accounts but an account can only have one "consultant" which is a user.

One way to design this would be a manager_id, sales_id, consultant_id field on the account table and then a many-to-many table with users for "owners".

Another way would be just one many-to-many table with users and accounts and then an (enum) "type" field on the pivot table.

How would you guys design this?

kiwid
Sep 30, 2013

Is there a way to add a "dynamic" column to a SELECT query based on another column?

For example, I have a simple table with a field called TicketNumber and a bunch of other fields that don't matter for the example.

The TicketNumber field has data like this:

L32342
M3232
N2132
L2132
A3232
L33265

The first letter of the ticket represents the branch it belongs to. No other fields in the table relate the branch, there is no branch_id or anything else.

I want to build a query that selects the Branch name and some other fields. Can I dynamically select a Branch field that looks at the first letter of the ticket number and sort of does a lookup in a key-value array or something?

I can do this in the software but I'd rather do it in the query.

edit: It's MySQL.

edit2: There is no branches table to join with which is why I'm resorting to this.

kiwid fucked around with this message at 16:15 on Jul 17, 2017

kiwid
Sep 30, 2013

Thanks, I ended up just saying gently caress it and created a branch table. Overkill for this application but oh well.

kiwid
Sep 30, 2013

With MySQL, is it possible to group by just certain groups and have everything else grouped as "Other"?

Example: I just want group 1 and 2 on their own and everything else grouped together.

Table:
group | quantity
--------------------
group1 | 5
group1 | 2
group3 | 4
group2 | 1
group1 | 6
group3 | 8
group4 | 5
group5 | 1

I want a result set like this where I sum the quantities:
group1 | 13
group2 | 1
other | 18

Possible with one query?

kiwid
Sep 30, 2013

Is there a way to do like a conditional group by (MySQL/MariaDB) on only certain values then group everything else into one row?

For example, I have a table of data like this:

code:
thing1	5
thing2	5
thing1	5
thing1	5
thing3	5
thing3	5
thing4	5
thing5	5
I want a result set that looks like this (only thing1 and thing2 get grouped on their own and everything else rolls up into "other"):
code:
thing1	15
thing2	5
other	20
I know I can use a UNION but wondering if it's possible without?

Unrelated------------------------------------------


What gui tool is this?

kiwid
Sep 30, 2013

kumba posted:

I know in MSSQL you can group by a case statement, can you do that in MYSQL?

pseudo-code:
code:
select thing, sum(value)
from mytable
group by
     case when thing not in ('thing1','thing2') then 'other'
     else thing
     end
order by thing
i think that works?

That works perfect, thanks!

Next question, I'm also using a group by with rollup. Is there a way to choose what name is on that rollup row?

Here is my actual query:

code:
SELECT
    CASE WHEN `product_subclass` IN ('IMAGERY', 'SAMPLING') THEN `product_subclass` ELSE 'OTHER' END AS `name`,
    ROUND(SUM(`previous_sales`), 2) AS `previous_sales`,
    ROUND(SUM(`previous_gp`), 2) AS `previous_gp`,
    ROUND(SUM(CASE WHEN `unit` = 'ACRE' THEN `previous_quantity` ELSE 0 END), 2) AS `previous_quantity`,
    ROUND((SUM(`previous_gp`) / SUM(`previous_sales`)) * 100, 2) AS `previous_percent`,
    ROUND(SUM(`current_sales`), 2) AS `current_sales`,
    ROUND(SUM(`current_gp`), 2) AS `current_gp`,
    ROUND(SUM(CASE WHEN `unit` = 'ACRE' THEN `current_quantity` ELSE 0 END), 2) AS `current_quantity`,
    ROUND((SUM(`current_gp`) / SUM(`current_sales`)) * 100, 2) AS `current_percent`
FROM `metrics`
INNER JOIN `locations` ON `metrics`.`location_id` = `locations`.`id`
WHERE `product_subgroup` = 'FERTILIZER.SERVICES'
    AND `product_class` = 'LARESCO'
    AND `locations`.`id` = '5'
GROUP BY
    CASE WHEN `product_subclass` IN ('IMAGERY', 'SAMPLING')
    THEN `product_subclass`
    ELSE 'OTHER' END ASC WITH ROLLUP
and the result set (notice the total row says "SAMPLING")

kiwid
Sep 30, 2013

kumba posted:

Try this:

code:
SELECT
    CASE WHEN COALESCE(`product_subclass`,'Grand Total') IN ('IMAGERY', 'SAMPLING','Grand Total') THEN COALESCE(`product_subclass`,'Grand Total') ELSE 'OTHER' END AS `name`,
    ROUND(SUM(`previous_sales`), 2) AS `previous_sales`,
    ROUND(SUM(`previous_gp`), 2) AS `previous_gp`,
    ROUND(SUM(CASE WHEN `unit` = 'ACRE' THEN `previous_quantity` ELSE 0 END), 2) AS `previous_quantity`,
    ROUND((SUM(`previous_gp`) / SUM(`previous_sales`)) * 100, 2) AS `previous_percent`,
    ROUND(SUM(`current_sales`), 2) AS `current_sales`,
    ROUND(SUM(`current_gp`), 2) AS `current_gp`,
    ROUND(SUM(CASE WHEN `unit` = 'ACRE' THEN `current_quantity` ELSE 0 END), 2) AS `current_quantity`,
    ROUND((SUM(`current_gp`) / SUM(`current_sales`)) * 100, 2) AS `current_percent`
FROM `metrics`
INNER JOIN `locations` ON `metrics`.`location_id` = `locations`.`id`
WHERE `product_subgroup` = 'FERTILIZER.SERVICES'
    AND `product_class` = 'LARESCO'
    AND `locations`.`id` = '5'
GROUP BY
    CASE WHEN `product_subclass` IN ('IMAGERY', 'SAMPLING')
    THEN `product_subclass`
    ELSE 'OTHER' END ASC WITH ROLLUP
That rollup row is NULL for Purchase type, so your case expression for the name is doing something funky with it I think. Give that a shot

e: you will probably have to fix some apostrophes because microsoft

hmm, that didn't seem to affect anything. That's ok though, I've worked around the problem. Thanks.

Unrelated question, I've been mulling the idea of switching from MySQL to Postgres. What is a good replacement GUI application for SQLyog?

kiwid
Sep 30, 2013

I keep being told I should stop using MySQL and use a "real" database.

What do people mean by this? I always ask for more clarification and then conversation stops.

kiwid
Sep 30, 2013

pangstrom posted:

They're probably either telling you to use PostgreSQL or a noSQL and you will know for sure which if you ask them which they use.

I believe they're referring to MSSQL, Oracle, and Postgres.

MSSQL and Oracle are two I'll likely never be interested in because I work primarily in PHP and Python. Should I however invest time into learning Postgres over MySQL?

kiwid fucked around with this message at 18:54 on Jul 17, 2018

kiwid
Sep 30, 2013

Ruggan posted:

Doesn’t MSSQL integrate just fine with PHP and Python? I think there are drivers for both.

Yeah but then I'd have to run a Windows Server.

kiwid
Sep 30, 2013

karms posted:

Kinda poo poo that he did not put a date on it, but oh well.

2013

https://github.com/grimoire-ca/bliki/blob/master/wiki/mysql/choose-something-else.md

kiwid
Sep 30, 2013

What is the best GUIs for postgres out there?

kiwid
Sep 30, 2013

Shy posted:

DataGrip is nice.

I've been eyeing this for a while. Guess I'll check it out.

kiwid
Sep 30, 2013

There must be something easy I'm missing about grouping data with non-aggregate columns.

My data looks like this:

code:
| employee_number | employee_first | employee_last | hours | type     | date       |
|-----------------|----------------|---------------|-------|----------|------------|
| 5               | John           | Smith         | 4     | regular  | 2020-11-02 |
| 5               | John           | Smith         | 8     | regular  | 2020-11-03 |
| 7               | Jane           | Doe           | 8     | regular  | 2020-11-05 |
| 5               | John           | Smith         | 2     | overtime | 2020-11-02 |
| 2               | Chris          | Simpson       | 8     | regular  | 2020-11-05 |
My query:

code:
select
    `employee_number`,
    sum(`hours`)
from `shifts`
where `type` = 'Regular'
group by `employee_number`
Is there anyway to include the first and last name columns in an aggregate query like this?

kiwid
Sep 30, 2013

min and max are the perfect solutions for my problem, I knew it'd be something simple. Much better than what I was doing which was select distinct employee_number, first, last then joining via a subquery in the from.

kiwid
Sep 30, 2013

Is there any decent courses out there under $100 that teach the ins and outs of SQL Server Management Studio?

kiwid
Sep 30, 2013

Been a while since I've had to write any SQL since being spoiled with an ORM.

I need to join a specific related row in a one-to-many relationship. For example:

orders
code:
| id |
|----|
| 1  |
| 2  |
| 3  |
instructions
code:
| order_id | type | notes |
|----------|------|-------|
| 1        | A    | blue   |
| 1        | B    | red   |
| 1        | C    | yellow   |
| 2        | A    | green   |
| 3        | C    | purple   |
I want a column in my result with only instructions with type A. For example:

code:
| id | notes |
|----|-------|
| 1  | blue  |
| 2  | green |
| 3  | null  |
Is the only way to do this by using a subquery on the join? Also type and order_id are a unique index and so there can't be multiple of the same type for an order.

kiwid
Sep 30, 2013

TheFluff posted:

If I'm understanding you correctly you can just use a left join and put the filter on instructions.type in the join clause instead of in the where clause, like so:

SQL code:
select
  orders.id,
  instructions.notes
from
  orders
  left join instructions on (
    orders.id = instructions.order_id
    and instructions.type = "A"
  )
For inner joins there is no difference between putting a condition in the join clause vs putting it in the where clause, but for left joins it matters - if you put instructions.type = "A" in the where clause instead you won't get order id 3 in the resultset.

Ah yes, this is exactly what I was looking for. Thanks.

kiwid
Sep 30, 2013

kiwid posted:

Been a while since I've had to write any SQL since being spoiled with an ORM.

I need to join a specific related row in a one-to-many relationship. For example:

orders
code:
| id |
|----|
| 1  |
| 2  |
| 3  |
instructions
code:
| order_id | type | notes |
|----------|------|-------|
| 1        | A    | blue   |
| 1        | B    | red   |
| 1        | C    | yellow   |
| 2        | A    | green   |
| 3        | C    | purple   |
I want a column in my result with only instructions with type A. For example:

code:
| id | notes |
|----|-------|
| 1  | blue  |
| 2  | green |
| 3  | null  |
Is the only way to do this by using a subquery on the join? Also type and order_id are a unique index and so there can't be multiple of the same type for an order.

So this query has evolved a bit and now I need a result that looks like this:

code:
| id | has_type_a      | has_type_b      | has_type_c      |
|----|-----------------|-----------------|-----------------|
| 1  | true            | true            | true            |
| 2  | true            | false (or null) | false (or null) |
| 3  | false (or null) | false (or null) | true            |
What would be the best way to achieve this now? I could join the instructions table for each column, I could do a subquery join, I also considered doing something like a group_concat() but the orders table has a large amount of non-aggregated columns I'm selecting that I haven't included in this example.

kiwid fucked around with this message at 18:43 on Jun 1, 2021

kiwid
Sep 30, 2013

Sorry I should have mentioned that yes I'm using T-SQL. I will investigate the pivot option. I definitely can't group because there are a lot of non-aggregated columns. Thanks.

edit: This is the current mess of a query that is joining the table multiple times:

SQL code:
SELECT orderlog.orderlog_nbr AS id,
    orderlog.orderlog_status,
    orderlog.container_release_number,
    orderlog.ship_booking_number,
    orderlog.plc_id AS location_id,
    location.short_name AS location_short_name,
    location.full_name AS location_name,
    orderlog.container_park,
    orderlog.ecn AS edn,
    orderlog.order_quantity,
    orderlog.order_quantity_uom,
    orderlog.nbr_of_containers,
    orderlog.packaging_type,
    orderlog.bag_weight_uom,
    orderlog.bag_markings,
    orderlog_bag_markings.notes AS pack,
    CASE 
        WHEN orderlog_nqac.orderlog_nbr IS NOT NULL
            THEN 'true'
        ELSE 'false'
        END AS nqac,
    CASE 
        WHEN orderlog_bag_phyto.orderlog_nbr IS NOT NULL
            THEN 'true'
        ELSE 'false'
        END AS phyto,
    CASE 
        WHEN orderlog_bag_sgs.orderlog_nbr IS NOT NULL
            THEN 'true'
        ELSE 'false'
        END AS sgs,
    CASE 
        WHEN orderlog_bag_strip_test.orderlog_nbr IS NOT NULL
            THEN 'true'
        ELSE 'false'
        END AS strip_test,
    CASE 
        WHEN orderlog_bag_grade_cert.orderlog_nbr IS NOT NULL
            THEN 'true'
        ELSE 'false'
        END AS grade_cert,
    orderlog.comments,
    orderlog.delivery_docket_pig_no,
    orderlog.delivery_docket_type,
    orderlog.commodity_id,
    commodity.name AS commodity_name,
    orderlog.iv_product AS product_id,
    product.description AS product_name,
    orderlog.vendor_id AS customer_id,
    vendor.short_name AS customer_short_name,
    vendor.full_name AS customer_name,
    orderlog.vendor_reference AS customer_reference,
    orderlog.ship_from_id,
    ship_from.short_name AS ship_from_short_name,
    ship_from.full_name AS ship_from_name,
    orderlog.ship_to_id,
    ship_to.short_name AS ship_to_short_name,
    ship_to.full_name AS ship_to_name,
    orderlog.shipping_id AS shipping_line_id,
    shipping_line.short_name AS shipping_line_short_name,
    shipping_line.full_name AS shipping_line_name,
    orderlog.vessel,
    orderlog.voyage,
    orderlog.loading_port,
    orderlog.discharge_port,
    orderlog.destination,
    orderlog.freight_carrier_id,
    freight_carrier.short_name AS freight_carrier_short_name,
    freight_carrier.full_name AS freight_carrier_name,
    orderlog.add_date,
    orderlog.start_date,
    orderlog.train_date,
    orderlog.etd,
    orderlog.cut_off_date
FROM production.dba.s1_orderlog AS orderlog
INNER JOIN production.dba.s1_name_and_address AS location ON orderlog.plc_id = location.name_and_address_id
INNER JOIN production.dba.s1_commodity AS commodity ON orderlog.commodity_id = commodity.commodity_id
INNER JOIN production.dba.s1_name_and_address AS vendor ON orderlog.vendor_id = vendor.name_and_address_id
INNER JOIN production.dba.s1_name_and_address AS ship_from ON orderlog.ship_from_id = ship_from.name_and_address_id
INNER JOIN production.dba.s1_name_and_address AS ship_to ON orderlog.ship_to_id = ship_to.name_and_address_id
LEFT OUTER JOIN production.dba.s1_name_and_address AS shipping_line ON orderlog.shipping_id = shipping_line.name_and_address_id
LEFT OUTER JOIN production.dba.s1_name_and_address AS freight_carrier ON orderlog.freight_carrier_id = freight_carrier.name_and_address_id
LEFT OUTER JOIN production.dba.s1_commodity_iv_product AS product ON orderlog.commodity_id = product.commodity_id
    AND orderlog.iv_product = product.iv_product
LEFT OUTER JOIN production.dba.s1_orderlog_loading_instr AS orderlog_bag_markings ON orderlog.plc_id = orderlog_bag_markings.plc_id
    AND orderlog.orderlog_nbr = orderlog_bag_markings.orderlog_nbr
    AND orderlog_bag_markings.loading_instruction_nbr = '54'
LEFT OUTER JOIN production.dba.s1_orderlog_loading_instr AS orderlog_nqac ON orderlog.plc_id = orderlog_nqac.plc_id
    AND orderlog.orderlog_nbr = orderlog_nqac.orderlog_nbr
    AND orderlog_nqac.loading_instruction_nbr = '44'
LEFT OUTER JOIN production.dba.s1_orderlog_loading_instr AS orderlog_bag_phyto ON orderlog.plc_id = orderlog_bag_phyto.plc_id
    AND orderlog.orderlog_nbr = orderlog_bag_phyto.orderlog_nbr
    AND orderlog_bag_phyto.loading_instruction_nbr = '10'
LEFT OUTER JOIN production.dba.s1_orderlog_loading_instr AS orderlog_bag_sgs ON orderlog.plc_id = orderlog_bag_sgs.plc_id
    AND orderlog.orderlog_nbr = orderlog_bag_sgs.orderlog_nbr
    AND orderlog_bag_sgs.loading_instruction_nbr = '7'
LEFT OUTER JOIN production.dba.s1_orderlog_loading_instr AS orderlog_bag_strip_test ON orderlog.plc_id = orderlog_bag_strip_test.plc_id
    AND orderlog.orderlog_nbr = orderlog_bag_strip_test.orderlog_nbr
    AND orderlog_bag_strip_test.loading_instruction_nbr = '9'
LEFT OUTER JOIN production.dba.s1_orderlog_loading_instr AS orderlog_bag_grade_cert ON orderlog.plc_id = orderlog_bag_grade_cert.plc_id
    AND orderlog.orderlog_nbr = orderlog_bag_grade_cert.orderlog_nbr
    AND orderlog_bag_grade_cert.loading_instruction_nbr = '17'
WHERE (orderlog.orderlog_status <> 'x')
    AND (orderlog.plc_id = '01')
    AND (orderlog.orderlog_type_code = '3')

kiwid fucked around with this message at 01:51 on Jun 2, 2021

kiwid
Sep 30, 2013

Heavy_D posted:

In the pivot approach, if you add something like:
code:
LEFT JOIN
(
   (SELECT order_id, instruction 
      FROM instructions) AS instr_rows
     PIVOT
   (MIN(instruction) 
    FOR instruction IN ([A],[B],[C])) AS instr_cols
)
ON instr_cols.order_id = orderlog.orderlog_nbr
You should have access to columns instr_cols.[A], instr_cols.[B] and instr_cols.[C] in your main query, which you can test for being null/non-null using a case statement in the same manner as your strip_test column.

Hey thanks,

PIVOT ended up being exactly what I needed. It took me a little to figure out the syntax but I've modified the query and it's working.

kiwid
Sep 30, 2013

Alright, so I've finally launched my application into production which is simply a web dashboard with a read only connection to our ERP software's database for reporting purposes. Unfortunately I'm seeing deadlocks now which didn't occur during development and so my select queries being cancelled. I've never experienced deadlocking before so I'm not sure how to come up with a solution here. It appears that when the ERP software is making update transactions, it'll lock the rows and won't allow me to issue a select statement. The reporting needs to be as "real-time" as possible and so an ETL process into a data warehouse wasn't the solution I determined a while back, at least in this instance.

Here is the error I'm seeing:

quote:

[2021-06-07 15:44:27] production.ERROR: SQLSTATE[40001]: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Transaction (Process ID 75) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

I started Googling and found this "NOLOCK" hint (https://www.mssqltips.com/sqlservertip/2470/understanding-the-sql-server-nolock-hint/). Just wondering if any of you have any feedback on using something like this? I understand the possibility of getting a "dirty read", but I believe that's acceptable considering the dashboard is polling an updated query every 60 seconds and would correct itself.

kiwid
Sep 30, 2013

Nth Doctor posted:

Our app is built around accepting dirty reads as okay, and squeezing every ounce of performance out of the database server. For years, using (NOLOCK) was a near-mandatory standard. Several years back, we discovered SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED aka STILRU which is basically batch level NOLOCK and is the norm now.

We know dirty reads are a thing, but in the interest of deadlock dodging, we accept the risks. It's not the greatest because of the reads issue, and a slightly more conservative SET TRANSACTION ISOLATION LEVEL SNAPSHOT ISOLATION.

Here's an article I skimmed: https://www.sqlshack.com/snapshot-isolation-in-sql-server/

Instant edit:
I would also do some profiling to see why your CRUD is longrunning enough to cause deadlock issues. Maybe its nothing and has to do with the locks that SELECTs are taking on underlying rows, but it's worth a look.

Thanks for this information. I'll implement NOLOCK as a quick fix then and I'll start to investigate the SNAPSHOT ISOLATION. I'll also look into the profiling. This database is 65GB so I'm sure there are some performance issues there but I'm not sure if it's server related or just lovely ERP software related. As a side note, is 65GB considered big these days?

kiwid
Sep 30, 2013

I'm building a view only dashboard that has related meta data on top of our main production database. I've created a separate database for the application with read/write, and I'm pulling in data from our production database using a view and a read only connection. I was considering adding foreign keys in my app's database relating the meta data to the data in the view but I started wondering if this would affect the production database in any way. Like it wouldn't prevent a delete or something on the production database would it?

kiwid
Sep 30, 2013

Does anyone know why when restoring a database via SSMS to a different database name, sometimes (seems to be based on the individual database) it doesn't auto-rename the files to restore as?

For example, one of my databases when restoring I simply change the destination database name and the file names are automatically changed as well:





However, with a different database this doesn't happen, I have to manually rename the files when restoring to a different destination database name:



Adbot
ADBOT LOVES YOU

kiwid
Sep 30, 2013

Anyone know why the backups for one of my databases has suddenly shrunk by a lot? This is SQL Server and the option for auto shrink is turned off.

I don't see any indication that the data has been affected in any way. Everything is still there.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply