Industries
|
|
|
Our friend Richard Frederick gave an excellent series of twenty talks on data from last August through this January, and I highly encourage you to review them at your convenience.
I've done this for a long time and I learned a lot from him.
The videos and PDFs from his talks may be found here.
Agile is Dead (in a rigorously formal sense)
Link to detailed discussion.
Link to detailed discussion.
The engagement is what we do to effect a change that serves customers.
The system is what we analyze and either build or change to serve customers.
The solution is the change we make to serve customers.
Links to detailed discussions.
There are contexts for each situation.
Discovery is a qualitative process. It identifies nouns (things) and verbs (actions, transformations, decisions, calculations).
It's how you go from this... |
...to this. |
|
![]() |
![]() |
![]() |
Discovery comes first, so you know what data you need to collect.
Elicitation is discovering the customer's needs. Discovery is about mapping the customer's existing process — or — it's what you design by working back from the desired outcomes.
Link to detailed discussion.
Data Collection is a quantitative process. It identifies adjectives (colors, dimensions, rates, times, volumes, capacities, materials, properties).
It's how you go from this... |
...to this. |
|
![]() |
![]() |
![]() |
Think about what you'd need to know about a car in the context of traffic, parking, service (at a gas station or border crossing), repair, insurance, design, safety, manufacturing, marketing, finance, or anything else.
![]() |
|
It doesn't happen by accident!
![]() |
|
Link to detailed discussion.
Link to detailed discussion.
There are many forms of this (rather chicken-and-egg) equation, but this one is fairly common:
n >= (z ⋅ σ / MOE)2
where:
n = minimum sample size (should typically be at least 30)
z = z-score (e.g., 1.96 for 95% confidence interval)
σ = sample standard deviation
MOE = measure of effectiveness (e.g., difference between sample and population means in units of whatever you're measuring)
Document all procedures and assumptions!
Link to detailed discussion.
Complete data may not be available, and for good reason. Keeping records is sometimes rightly judged to be less important than accomplishing other tasks.
Here are some options* for dealing with missing data (from Udemy course R Programming: Advanced Analytics In R For Data Science by Kirill Eremenko):
* These mostly apply to individual values in larger data records.
Document all procedures and assumptions!
Data from different sources may need to be regularized so they all have the same units, formats, and so on. (This is a big part of ETL efforts.)
Sanity checks should be performed for internal consistency (e.g., a month's worth of hourly totals should match the total reported for the month).
Conversely, analysts should be aware that seasonality and similar effects mean subsets of larger collections of data may vary over time.
Data items should be reviewed to see if reporting methods or formats have changed over time.
Data sources should be documented for points of contact, frequency of issue, permissions and sign-offs, procedures for obtaining alternate data, and so on.
...as data is becoming more voluminous, and as what can be done with it is more powerful and valuable.
This presentation and other information can be found at my website:
E-mail: bob@rpchurchill.com
LinkedIn: linkedin.com/in/robertpchurchill