Questions tagged [database]

A comprehensive collection of related data organized for convenient access, generally associated with software to update and query the data.

Filter by
Sorted by
Tagged with
1
vote
1answer
54 views

Newbie Data Analyst Questions

I have recently applied to a Job by a startup that wants to higher a data analyst. I have been going through the rounds and it seems like they want to hire me for the position. I have recently ...
1
vote
0answers
10 views

Sources for panel data on German house prices [closed]

I'm looking for sources for panel data, preferably a database, on German house prices, it doesn't have to be free, but nothing expensive. I've tried the federal government database, but I just can't ...
0
votes
0answers
13 views

CRSP/Compustat - Issue with gaps in data

I would like to test empirically CAPM for my dissertation. I have downloaded monthly data for period 01-1990 - 12-2018 for NYSE from CRSP/Compustat database using condition "exchangecode" = 1 however, ...
2
votes
0answers
17 views

Probability: Estimating Database Size with N Smallest Random Values

Note: This was first posted on StackOverflow, but I did not have much luck there. I have a very large database. Each item in the database is assigned a uniformly distributed random value $\geq 0$ and ...
0
votes
0answers
12 views

Questions related to an upcoming databse project

Starting this summer, I will work on a project where I am supposed to create a database in Excel. Without going into much detail, it would be nice to know if anyone has done something similiar to see ...
1
vote
1answer
33 views

How to perform specific queries in weather data time series [closed]

I have time series data from several weather stations located in a specific area. The readings include a timestamp, the humidity and the temperature. The resolution of the data is quite high, about 6 ...
2
votes
0answers
64 views

Which duration model to use with a fully right censored database

Currently I'm examining the duration of residence of households. I have a database at my disposal that indicates how long a specific household resides in its current home. I want to explain their ...
1
vote
0answers
211 views

Are sqlite databases more efficient (in terms of RAM usage) than the dplyr library in R?

I have to perform a left join on two large datasets. My understanding of dplyr is that it duplicates (temporarily) the two datasets in memory (or something similar which generates a large temporary ...
2
votes
1answer
96 views

Why does “sticky noise” defy averaging attack?

I have read an interesting paper (pdf) describing how a privacy preserving technique might be breached, but I am having trouble understanding the following paragraph describing one of several layers ...
1
vote
1answer
58 views

An open database for propensity score matching

I need an open biological or medical database for benchmarking methods for propensity score matching. Where can I find some?
0
votes
1answer
147 views

Recommend a DB to process large data from MySQL [closed]

Basically, right now I'm working with data that has reached 1+ TB on MySQL and I need to run a statistical algorithm on it in the future. I also need to process the data as well. What I'm hoping to ...
1
vote
1answer
25 views

Salinity affecting Plant Growth

I conducted an experiment where I treated 9 bean seeds to different levels of salinity (ranging from 0g to 4g of salt) when watering them, in order to determine whether if salinity affected seed ...
1
vote
1answer
86 views

What is the best way to prepare a database export to a statistician for statistical analysis?

I work in Rome for the Italian National Health Service (www.iss.it); I am currently working on a European clinic project called EMTICS (www.emtics.eu), I developed both the database and the website to ...
3
votes
2answers
2k views

Statistical analysis of relational database: is it possible and how?

I have been struggling with flat file databases and corresponding statistical packages for almost 20 years now (from Excel to SPSS, then Stata, and currently R). However, I have always had to convert ...
0
votes
0answers
38 views

Database and Data Waehouse design and processing in Big Data

With the growing popularity of Big Data and related tools, I am looking for books that specifically deal with implementing a warehouse in one of Big Data technologies, database and data-warehouse ...
4
votes
1answer
1k views

What is the purpose of using a Laplacian distribution in adding noise for Differential Privacy?

I am reading up on Differential Privacy and it is mentioned that the technique relies on adding some controlled noise to the release of responses to queries towards a statistical database. This is ...
0
votes
1answer
337 views

How could i use r to add data to a table? [closed]

I am looking for an approach. We have a historical table with the consumption per week of a list of items (19.000 item). We want to predict the consumption for the next n weeks. We are thinking on ...
5
votes
2answers
393 views

How to make an effective sampling from a database of text documents?

Problem: I want to know methods to perform an effective sampling from a database. The size of the database is about 250K text documents and in this case each text ...
1
vote
1answer
6k views

What is the difference (if any) between 'harmonized' and 'standardized data? [closed]

I am trying to describe the process of combining multiple datasets into a common format - e.g. a single database that enforces a common vocabulary, scale, and structure. However, I am unclear if I ...
1
vote
1answer
259 views

Recommendation about what statistical test to use? Impact of regulatory change

I have a problem in determining the right statistical test for my research. First I will give some background information. I want to research the impact of IFRS (regulatory change) on audit fees (fees ...
0
votes
1answer
832 views

How to apply an R prediction model to very big data from SQL database in parallel.

I dont need to load the entire dataset into memory. In fact I only need 1 row at a time to apply a trained model, get the predicted response and put that response somewhere, possibly back into another ...
-3
votes
1answer
49 views

I am wondering what model to use [closed]

I am trying to do an analysis of popularity growth, in particular of the rate of growth in play count for artists on an online music streaming service. I want to create a postgresql db filled with ...
1
vote
1answer
166 views

I would like to analyze the data using R and Python. How to handle 10 millions records of data? [closed]

I would like to analyze the data using R and Python, and I am unable to open text file due to huge number of records. Is there any free database software to install in PC and load the data into it and ...
0
votes
0answers
16 views

Mathematics of simple performance testing [duplicate]

I have a set of sorted tables T that have known but different dimensions. There are two types of functions in this system: f(T) g(T, n), where n is an integer parameter. ... and two types of costs ...
2
votes
1answer
52 views

Is Outlier detection in two separate databases is equal to one combined database?

Suppose that we have two databases : Database_1 and Database_2 . Database_1 has 300 samples ...
1
vote
1answer
33 views

What is a nonreductive database?

A database like Genbank is said to be a nonreductive database. What does that mean ?
0
votes
2answers
644 views

Any suggestions for storing diverse data sets for on-line visualization?

We are trying to design a data repository for storing datasets, from many different sources, in such a way that they can easily be used in online data presentations. This data repository will be part ...
0
votes
1answer
111 views

Freeware similar to MS Access for SQL-less data mining, querying and reporting [closed]

I am looking for freeware similar to MS Access for data mining, querying and reporting without really writing SQL (which I know very well but sometimes I feel lazy and would rather just drag and drop)....
0
votes
1answer
28 views

Dealing with a variable amount of entries for a given attribute

I want to analyze a .csv file with the results of a customer survey. One of the questions is: "Tell us the top 3 reasons you chose us.". The answers are short paragraphs of text listing a ranging ...
1
vote
1answer
389 views

Statistical package that works with sql database [closed]

At my firm we have been calculating fundamental stock factors each week for over a decade. We have the performance of each of these factors. I'd like to enhance the statistical analysis of these ...
3
votes
0answers
783 views

A non-technical visualization of my database (a layman's ER Diagram?) [closed]

I would like to provide an intuitive visualization of my database for readers and users of the database that does not require technical knowledge of database structure. The figure will be included in ...
2
votes
2answers
584 views

Organizing cluster analysis results in a database

I'm a newbie in cluster analysis so please excuse me if my question seems to be very basic. I'm using SPSS and Matlab for performing cluster analysis in a variety of datasets. Dendograms are great for ...
5
votes
3answers
3k views

Suggestions on how to merge multiple datasets with an imperfect ID across databases?

I have four databases of books that I have assembled from various sources, websites, etc. I would like to merge the databases, but I face a significant merging issue in that there is no "perfect" ...
4
votes
2answers
336 views

Public databases of learned HMM models for NLP

I understand that HMM models model language with Parts of Speech (POS) as hidden states and words as observations. These HMM models are usually learned from large text corpora, and many of these ...
18
votes
2answers
11k views

Quality assurance and quality control (QA/QC) guidelines for a database

Background I am overseeing the input of data from primary literature into a database. The data entry process is error prone, particularly because users must interpret experimental design, extract ...